CN117609652A

CN117609652A - Display equipment and webpage screen projection method

Info

Publication number: CN117609652A
Application number: CN202310747552.8A
Authority: CN
Inventors: 张敬坤; 庞秀娟
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2023-06-21
Filing date: 2023-06-21
Publication date: 2024-02-27

Abstract

The application provides a display device and a screen projection method of a webpage, wherein the method comprises the following steps: responding to a device discovery request sent by the mobile terminal based on a screen-throwing protocol, and expanding the screen-throwing protocol based on the device discovery request so that the screen-throwing protocol supports the screen-throwing event of the webpage; responding to a screen throwing request of a screen throwing webpage sent by a mobile terminal, and acquiring a screen throwing address input by the mobile terminal; loading the screen throwing address to generate a screen throwing picture of the screen throwing webpage, and controlling the display to display the screen throwing picture in the user interface through the webpage view control. The display device can combine the mobile terminal, the screen-throwing webpage and the large screen of the display device together to acquire the screen-throwing address input by the mobile terminal, loads the screen-throwing address through the webpage view control, realizes the effect of watching webpage resources through the large screen, synchronously displays and responds to input contents at the large screen end through the synchronous control mechanism of the display device and the mobile terminal, and further solves the problem that the display device cannot realize the screen throwing of the webpage.

Description

Display equipment and webpage screen projection method

Technical Field

The application relates to the technical field of screen projection, in particular to a screen projection method of display equipment and a webpage.

Background

The display device may screen a screen displayed in a screen-casting initiating device such as a mobile terminal to a screen-casting receiving device such as a display device due to the property of the larger screen size. For example, the screen of a mobile terminal such as a mobile phone is smaller, viewing experience can be affected when viewing media assets, and for the scene, the media assets such as pictures and videos played in the mobile terminal can be projected to a television, so that the display device can display screen projection content corresponding to screen projection data sent by the mobile terminal in real time, and the viewing experience of a user is improved through a large screen of the display device.

In the process of executing the screen projection, the screen projection operation can be realized in various modes. For example, the screen casting may be implemented based on a digital living alliance (Digital Living Network Alliance, DLNA) protocol, hereinafter referred to as DLNA protocol. The DLNA protocol has the main content of realizing media resource sharing between the screen throwing initiating equipment and the screen throwing receiving equipment. For example, pictures or videos and other media in mobile terminals such as mobile phones can be delivered to display devices such as televisions through some interactive instructions, and display and play are performed through the display devices.

However, the DLNA protocol only supports the formats of pictures, audio and video, and does not support the screen dropping of web pages. When the mobile terminal executes web browsing, web screen projection cannot be realized at the display equipment side. When the user wants to watch the webpage content displayed by the mobile terminal through the large screen, the user can log in the personal account number and synchronously watch the webpage content with the mobile terminal in a mode of logging in the webpage at the display equipment end. However, when the display device loads the webpage through the large screen, the input method is complex in inputting the content in the webpage through the large screen, the input method is inconvenient to operate, and the focus of the remote controller is inconvenient to control, that is, the problem of inconvenient operation exists when the webpage is loaded only through the display device, and the method is inconvenient for a user to operate the browser webpage. In summary, when the mobile terminal displays the webpage to be projected, the display device end cannot realize the projection of the webpage to be projected.

Disclosure of Invention

Some embodiments of the present application provide a display device and a method for projecting a web page, so as to solve the problem that a display device end cannot realize the projection of the web page.

In a first aspect, some embodiments of the present application provide a display device, including:

a display configured to display a user interface;

a communicator configured to establish a screen-drop connection with the mobile terminal;

a controller configured to:

responding to a device discovery request sent by a mobile terminal based on a screen-throwing protocol, and expanding the screen-throwing protocol based on the device discovery request so that the screen-throwing protocol supports a screen-throwing event of a webpage;

responding to a screen throwing request of a screen throwing webpage sent by a mobile terminal, and acquiring a screen throwing address input by the mobile terminal;

loading the screen-throwing address to generate a screen-throwing picture of the screen-throwing webpage;

and controlling the display to display the screen throwing picture in a user interface through a webpage view control.

In a second aspect, some embodiments of the present application provide a method for projecting a web page, which may be applied to the display device of the first aspect, where the method for projecting a web page includes:

As can be seen from the above technical solutions, some embodiments of the present application provide a display device and a method for projecting a web page, where the method includes: responding to a device discovery request sent by the mobile terminal based on a screen-throwing protocol, and expanding the screen-throwing protocol based on the device discovery request so that the screen-throwing protocol supports the screen-throwing event of the webpage; responding to a screen throwing request of a screen throwing webpage sent by a mobile terminal, and acquiring a screen throwing address input by the mobile terminal; loading the screen throwing address to generate a screen throwing picture of the screen throwing webpage, and controlling the display to display the screen throwing picture in the user interface through the webpage view control. The display device can combine the mobile terminal, the screen-throwing webpage and the large screen of the display device together to acquire the screen-throwing address input by the mobile terminal, loads the screen-throwing address through the webpage view control, realizes the effect of watching webpage resources through the large screen, synchronously displays and responds to input contents at the large screen end through the synchronous control mechanism of the display device and the mobile terminal, and further solves the problem that the display device cannot realize the screen throwing of the webpage.

Drawings

In order to more clearly illustrate some embodiments of the present application or technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

fig. 2 is a block diagram of a hardware configuration of a display device 200 provided in some embodiments of the present application;

FIG. 3 is a block diagram of a hardware configuration of a control device provided in some embodiments of the present application;

fig. 4 is a schematic diagram of software configuration in a display device according to some embodiments of the present application;

fig. 5 is a flowchart of a method for performing screen projection of a web page by a display device according to some embodiments of the present application;

fig. 6 is a schematic flow chart of a display device according to some embodiments of the present application, which extends a screen-casting protocol based on a device discovery request;

fig. 7 is a schematic flowchart of a display device according to some embodiments of the present application to obtain a screen address input by a mobile terminal;

Fig. 8 is a schematic view of a scenario in which a mobile terminal provided in some embodiments of the present application interacts with a display device;

fig. 9 is a flowchart of a screen shot screen of a screen shot webpage generated by loading a screen shot address by a display device according to some embodiments of the present application;

fig. 10 is a schematic view of a screen effect provided in some embodiments of the present application;

FIG. 11 is a schematic view of another screen effect according to some embodiments of the present disclosure;

fig. 12 is a flowchart illustrating a control of a screen-throwing webpage by a display device according to some embodiments of the present application;

fig. 13 is a schematic view of a scene of a touch screen webpage of a display device according to some embodiments of the present application through a virtual remote control operation type or a sliding operation type;

fig. 14 is a schematic flow chart of performing input synchronization between a display device and a mobile terminal according to some embodiments of the present application;

fig. 15 is a flowchart of a method for projecting a web page according to some embodiments of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of some embodiments of the present application more clear, the technical solutions of some embodiments of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application.

It should be noted that the brief description of the terms in some embodiments of the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the implementation of some embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms first, second, third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in fig. 1, a user may operate the display device 200 through the mobile terminal 300 and the control device 100.

In some embodiments, the mobile terminal 300 may install a software application with the display device 200, implement connection communication through a network communication protocol, and achieve the purpose of one-to-one control operation and data communication. The audio/video content displayed on the mobile terminal 300 can also be transmitted to the display device 200, so as to realize the synchronous display function.

As also shown in fig. 1, the display device 200 is also in data communication with the server 400 via a variety of communication means. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks.

The display apparatus 200 may additionally provide a smart network television function of a computer support function, including, but not limited to, a network television, a smart television, an Internet Protocol Television (IPTV), etc., in addition to the broadcast receiving television function.

Fig. 2 is a block diagram of a hardware configuration of a display device 200 provided in some embodiments of the present application.

In some embodiments, display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, memory, a power supply, a user interface.

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, for receiving an image signal from the controller output, for displaying video content, image content, and components of a menu manipulation interface, and a user manipulation UI interface, etc.

In some embodiments, communicator 220 is a component for communicating with external devices or servers 400 according to various communication protocol types.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200.

In some embodiments, a user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI).

In some embodiments, user interface 280 is an interface that may be used to receive control inputs.

Fig. 3 is a block diagram of a hardware configuration of a control device according to some embodiments of the present application. As shown in fig. 3, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface, a memory, and a power supply.

The control device 100 is configured to control the display device 200, and can receive an input operation instruction of a user, and convert the operation instruction into an instruction recognizable and responsive to the display device 200, functioning as an interaction between the user and the display device 200.

In some embodiments, the control device 100 may be a smart device. Such as: the control apparatus 100 may install various applications for controlling the display apparatus 200 according to user's needs.

In some embodiments, as shown in fig. 1, a mobile terminal 300 or other intelligent electronic device may function similarly to the control device 100 after installing an application that manipulates the display device 200.

The controller 110 includes a processor unit 112 and RAM 113 and ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication collaboration among the internal components and the external and internal data processing functions.

The communication interface 130 enables communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of a WiFi chip 131, a bluetooth module 132, an NFC module 133, and other near field communication modules.

A user input/output interface 140, wherein the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, keys 144, and other input interfaces.

In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input-output interface 140. The control device 100 is provided with a communication interface 130 such as: the WiFi, bluetooth, NFC, etc. modules may send the user input instruction to the display device 200 through a WiFi protocol, or a bluetooth protocol, or an NFC protocol code.

A memory 190 for storing various operation programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 may store various control signal instructions input by a user.

A power supply 180 for providing operating power support for the various elements of the control device 100 under the control of the controller.

Fig. 4 is a schematic view of software configuration in a display device according to some embodiments of the present application, in some embodiments, the system is divided into four layers, namely, an application layer (application layer), an application framework layer (Application Framework layer), a system library layer (system runtime layer), and a kernel layer from top to bottom.

In some embodiments, at least one application program is running in the application program layer, and these application programs may be a Window (Window) program of an operating system, a system setting program, a clock program, a camera application, and the like; or may be an application developed by a third party developer.

The framework layer provides an application programming interface (Aplication Pogramming Iterface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions. The application framework layer corresponds to a processing center that decides to let the applications in the application layer act.

As shown in fig. 4, the application framework layer in some embodiments of the present application includes a manager (manager), a Content Provider (Content Provider), a View System (View System), and the like. Wherein the manager comprises at least one of the following modules: an Activity Manager (Activity Manager) is used to interact with all activities that are running in the system; a Location Manager (Location Manager) is used to provide system services or applications with access to system Location services; a Package Manager (Package Manager) for retrieving various information about an application Package currently installed on the device; a notification manager (Notification Manager) for controlling the display and clearing of notification messages; a Window Manager (Window Manager) is used to manage bracketing icons, windows, toolbars, wallpaper, and desktop components on the user interface.

In some embodiments, the activity manager is to: managing the lifecycle of the individual applications and typically the navigation rollback functionality.

In some embodiments, a window manager is used to manage all window programs.

In some embodiments, the system runtime layer provides support for the upper layers, namely the framework layer, and when the framework layer is accessed, the operating system runs the C/C++ libraries contained in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the kernel layer contains at least one of the following drivers: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, touch sensor, pressure sensor, etc.), and the like.

In some embodiments, the kernel layer further includes a power driver module for power management.

In some embodiments, the software programs and/or modules corresponding to the software architecture in fig. 4 are stored in the first memory or the second memory shown in fig. 2 or fig. 3.

In some embodiments, the display device 200 may screen a screen displayed in a screen-casting initiating device such as the mobile terminal 300 into a screen-casting receiving device such as the display device 200 due to its larger screen size. For example, a mobile terminal 300 such as a mobile phone may have a smaller screen, and viewing media may affect the viewing experience. Aiming at the scene, the screen of the media such as pictures, videos and the like played in the mobile terminal 300 can be projected to the display device 200, so that the display device 200 can display the screen projection content corresponding to the screen projection data sent by the mobile terminal 300 in real time, and the viewing experience of a user is improved through the large screen of the display device 200.

In the process of executing the screen projection, the screen projection operation can be realized in various modes. For example, the projection may be implemented based on the digital living alliance DLNA protocol. The DLNA protocol has the main content of realizing media resource sharing between the screen throwing initiating equipment and the screen throwing receiving equipment. For example, pictures or videos or the like in the mobile terminal 300 such as a mobile phone or the like may be delivered to the display device 200 such as a television or the like through some interactive instructions, and display and playback are performed through the display device 200.

Since the screen of the mobile terminal 300 is small, when a user views pictures, videos, or browses web pages through the screen of the mobile terminal 300, the user's viewing experience may be affected. To enhance the viewing experience, content in the mobile terminal 200 may be projected into the display device 200 to view various media through a large screen of the display device 200. However, the DLNA protocol only supports the formats of pictures, audio and video, and does not support the screen dropping of web pages. Thus, when the mobile terminal 300 performs web browsing, web projection cannot be implemented at the display device 200.

In some embodiments, when the user wants to view the web page content displayed by the mobile terminal 300 through the large screen, the user logs in the personal account number and views the web page synchronously with the mobile terminal 300 by logging in the web page on the display device 200. For example, when the mobile terminal views the content of the application a, because the screen is smaller, the mobile terminal can input the website corresponding to the application a at the display device end and then log in the personal account number of the user, so that the content in the application a can be played synchronously with the mobile terminal at the display device end. However, when the display device 200 loads the web page through the large screen, the input method is complex when inputting the content in the web page through the large screen, and the input method is inconvenient to operate, and the focus of the remote controller is inconvenient to control, that is, the problem of inconvenient operation only when loading the web page through the display device 200 exists, and the operation of the browser web page by the user is inconvenient in this way. In summary, when the mobile terminal 300 displays the web page to be projected, the display device 200 cannot project the web page to be projected.

In order to solve the problem that the display device 200 cannot realize the screen projection of the webpage, some embodiments of the present application provide a display device 200. The display device 200 includes a display 260, a communicator 220, and a controller 250. Wherein the display 260 is configured to display a user interface, the communicator 220 is configured to establish a screen-casting connection with the mobile terminal 300, and the controller 250 is configured to perform a screen-casting method of a web page. In order to facilitate understanding of the technical solutions in some embodiments of the present application, the following details of each step are described with reference to some specific embodiments and the accompanying drawings.

Fig. 5 is a flowchart of a method for performing screen dropping of a web page by a display device according to some embodiments of the present application, as shown in fig. 5, in some embodiments, when the display device 200 performs the method for performing screen dropping of a web page, the method may include the following steps S1 to S4, which are specifically as follows:

step S1: in response to a device discovery request issued by the mobile terminal 300 based on the screen-casting protocol, the display device 200 expands the screen-casting protocol based on the device discovery request such that the screen-casting protocol supports screen-casting events of the web page.

In order for the screen-casting protocol to support the screen-casting event of the web page, in response to a device discovery request issued by the mobile terminal 300 based on the screen-casting protocol, the display device 200 may extend the screen-casting protocol based on the device discovery request. In some embodiments, the screen-throwing protocol may be a digital living alliance DLNA protocol, and the existing functions and supported types of the DLNA protocol are known, and when the supported types of the DLNA protocol are extended, the supported types of the web screen throwing can be supplemented on the original basis, so that the web screen throwing function from the mobile terminal 300 to the display device 200 can be realized.

Fig. 6 is a schematic flow chart of an extended screen-casting protocol of a display device according to some embodiments of the present application, as shown in fig. 6, in some embodiments, when the display device 200 extends the screen-casting protocol, the display device 200 may first generate a description file, where the description file is used to store a screen-casting type supported by the display device 200 and a control type for a screen-casting address, then add a preset field for supporting screen casting of a web page to the description file, extend the description file based on the preset field, generate an extended description file, and finally extend the screen-casting protocol according to the extended description file, so that the screen-casting protocol supports screen casting of the web page.

For example, the display device 200 may generate the description file after receiving a device discovery request issued by the mobile terminal 300 based on a screen-casting protocol. For example, the description file may be description. Xml, which is used to store the type of screen supported by the display device 200 and the type of control over the screen address. After the description file description.xml is obtained, adding a preset field for supporting webpage screen projection, such as a SCPDURL field, into the description file, expanding the description file description.xml based on the SCPDURL field to generate an expanded description file, and finally expanding a screen projection protocol, such as a DLNA protocol, according to the expanded description file, so that the screen projection protocol supports webpage screen projection.

By expanding the screen projection protocol, the screen projection protocol can support the function of webpage screen projection, so that when the mobile terminal 300 performs screen projection with the display device 200 based on the expanded screen projection protocol, the function of webpage screen projection from the mobile terminal 300 to the display device 200 can be realized, for example, the function of webpage screen projection from the mobile phone end to the television end with a large screen can be realized. That is, the application can realize the effect of the screen projection of the webpage by expanding the screen projection protocol, such as the DLNA protocol, and executing the device discovery through the screen projection protocol, thereby solving the problem that the screen projection of the webpage cannot be realized at the display device 200 end. After the completion of the execution of step S1, the following step S2 may be executed.

Step S2: in response to a screen-throwing request for a screen-throwing webpage sent by the mobile terminal 300, the display device 200 acquires a screen-throwing address input by the mobile terminal 300.

In order to acquire the screen address input by the mobile terminal 300, in some embodiments, the display device 200 may perform the following procedure. Fig. 7 is a schematic flowchart of a display device for acquiring a screen address input by a mobile terminal according to some embodiments of the present application, as shown in fig. 7, when the display device 200 acquires a screen address input by the mobile terminal 300, it may first receive a screen address transmission instruction sent by the mobile terminal 300, then parse the screen address transmission instruction, generate a uniform resource identifier carried in the screen address transmission instruction, and then convert the uniform resource identifier into a screen address.

Fig. 8 is a schematic view of a scenario in which a mobile terminal provided in some embodiments of the present application interacts with a display device, as shown in fig. 8, in some embodiments, when the mobile terminal 300 interacts with the display device 200, an application in the mobile terminal 300 may receive a multicast packet at the end of the display device 200, for example, a Nodify multicast packet, based on active or passive discovery of a projection protocol, such as a DLNA protocol, or the mobile terminal 300 may send an M-Search multicast packet to the display device 200 by itself. In some embodiments, a device type description, a search mode, a network connection status, etc. may be included in the multicast packet. The specific content may be set according to actual requirements, which is not limited in this application.

In some embodiments, the mobile terminal 300 may acquire relevant device information in the display device 200, for example, supported screen-cast types, supported interactive functions, etc., through the mobile terminal 300 transmitting or receiving a multicast packet, so that the mobile terminal 300 and the display device 200 may perform interaction based on the relevant device information.

After receiving the M-Search multicast packet transmitted from the mobile terminal 300 or transmitting the Nodify multicast packet to the mobile terminal 300, the display device 200 may transmit a unicast packet including the extension description file to the mobile terminal 300, so that the mobile terminal 300 acquires the screen-casting type supported by the display device 200 by parsing the unicast packet. Meanwhile, in response to the screen-casting type supported by the display device 200 including the web page screen-casting, the display device 200 may feed back a screen-casting reminder to the mobile terminal 300, so that the mobile terminal 300 searches the display device 200 in the same network according to the screen-casting reminder, and sends a screen-casting request for the screen-casting web page to the display device 200.

In some embodiments, in response to a screen-throwing request for a screen-throwing webpage sent by the mobile terminal 300, the display device 200 may acquire a screen-throwing address input by the mobile terminal 300, that is, after the mobile terminal 300 performs device discovery and selects the display device 200 to be thrown, the mobile terminal 300 may input the screen-throwing address, and the display device 200 may acquire the screen-throwing address input by the mobile terminal 300 and may display content corresponding to the screen-throwing address.

Illustratively, after the mobile terminal 300 inputs the URL address of the screen, the display device 200 may receive the URL address. For example, the URL address input in the mobile terminal 300 may be transmitted to the display device 200 through a service interface, may be input in the form of a click bookmark, or may be acquired in other manners. For another example, the URL may be pushed to the display device 200, or the display device 200 may actively obtain the URL, which is not limited in this application. After the completion of the execution of step S2, the following step S3 may be executed.

Step S3: the display device 200 loads the screen-cast address to generate a screen-cast screen of the screen-cast web page.

After the display device 200 obtains the screen-throwing address input by the mobile terminal 300, the screen-throwing address can be loaded, and after loading is completed, a screen-throwing picture corresponding to the screen-throwing webpage is generated. Fig. 9 is a schematic flow chart of loading a screen-throwing address by a display device to generate a screen-throwing picture of a screen-throwing webpage according to some embodiments of the present application, as shown in fig. 9, in some embodiments, when the display device 200 generates the screen-throwing picture of the screen-throwing webpage, the following manner may be implemented. Firstly, a play window can be created through a webpage view control of an application component, the play window is used for loading a screen throwing address, then a user agent identification is set in the play window, and finally a screen throwing picture of the screen throwing address is generated based on the screen throwing address and the user agent identification.

In some embodiments, the application component may be an Activity component for supporting interaction of a screen between the mobile terminal 300 and the display device 200, and the web page view control may be a WebView control, which is a control for displaying a web page. In connection with step S2, the uniform resource identifier may be Uniform Resource Identifier, i.e. a URI, so that after receiving the screen-throwing address transmission instruction of the mobile terminal 300, the display device 200 may parse the uniform resource identifier URI carried in the screen-throwing address transmission instruction, then convert the uniform resource identifier URI into a URL, and load the URL address through WebView of the Activity component.

To enhance the display effect of the on-screen web page in the display device 200 and make the on-screen web page more suitable for a large screen of the display device 200, in some embodiments, the User Agent identifier may be a User-Agent identifier, or UA identifier for short. Fig. 10 is a schematic view of a screen effect provided by some embodiments of the present application, and fig. 11 is a schematic view of another screen effect provided by some embodiments of the present application, as shown in fig. 10 and fig. 11, a display device 200 may create a play window through a web page view control of an application component, and then a User Agent identifier, such as a User-Agent, may be set in the play window, where the User Agent identifier may determine which form to load a screen webpage, and by analyzing a device of the display device 200, adapt to a loading mode suitable for a large screen, so that a presentation effect of the screen webpage is more matched with the large screen. And finally, generating a screen projection picture of the screen projection address based on the screen projection address and the User-Agent identifier. That is, after the display device 200 loads the URL address through WebView, the web resource corresponding to the URL address and matching with the large screen can be loaded through setting the User-Agent identifier of the play window. After the completion of the execution of step S3, the following step S4 may be executed.

Step S4: the display device 200 controls the display 260 to display a screen shot in the user interface through a web page view control.

To implement a screen-cast web page on a large screen in the display device 200, the display device 200 may control the display 260 to display the screen-cast screen in the user interface through the web page view control after generating the screen-cast screen of the screen-cast web page. In this way, the mobile terminal 300, the screen-throwing webpage and the large screen of the display device 200 can be combined together, the effect of watching webpage resources through the large screen is achieved, the watching experience of a user is improved, and the problem that the display device 200 cannot realize the screen throwing of the webpage is solved.

To achieve synchronous control of the screen between the mobile terminal 300 and the display device 200, in some embodiments, the display device 200 may send an operation instruction for controlling the screen-throwing webpage to the mobile terminal 300, and then monitor an operation event of the mobile terminal 300 on the screen-throwing webpage based on the operation instruction, generate a webpage control instruction according to the operation event, and execute the webpage control instruction on the screen-throwing webpage.

For example, in the screen-throwing process, it is necessary to ensure the synchronicity of the control of the screen-throwing webpage by the mobile terminal 300 and the display device 200, and to achieve this effect, the display device 200 may send an operation instruction for controlling the screen-throwing webpage to the mobile terminal 300 so as to notify the mobile terminal 300 that the control event for the screen-throwing webpage can be executed. Then, the operation event of the mobile terminal 300 on the screen-throwing webpage is monitored, a webpage control instruction is generated based on the operation event, and then the display device 200 executes the webpage control instruction on the screen-throwing webpage. For example, the mobile terminal 300 performs an enlarging operation on the screen-throwing webpage, and the display apparatus 200 may generate an enlarged control instruction based on an event of the enlarging operation, and then synchronously perform the enlarged control instruction on the screen-throwing webpage of the large screen, so that consistency of a screen of the screen-throwing webpage played by the mobile terminal 300 and a screen-throwing screen of the display apparatus 200 may be ensured.

In order to achieve the effect of the consistency of the screen-cast web page played by the mobile terminal 300 and the screen-cast screen of the display device 200, in some embodiments, the display device 200 may also be implemented as follows. Fig. 12 is a flowchart of a display device controlling a screen-drop web page according to some embodiments of the present application, as shown in fig. 12, the display device 200 may first establish a synchronous control mechanism between the mobile terminal 300 and the display device 200, and then, in response to a focus operation event of the user on the screen-drop web page at the mobile terminal 300, the display device 200 may obtain a focus position input by the user in the screen-drop web page of the mobile terminal 300, and synchronize the focus position in the screen-drop web page of the display device 200. And finally, generating a focus operation instruction according to the focus position and the focus operation event, and executing the focus operation instruction on the screen-throwing webpage.

For example, after receiving an operation instruction for controlling the screen-cast web page transmitted from the display device 200, the mobile terminal 300 may control the screen-cast web page through a virtual remote controller operation type or through a sliding operation type. Fig. 13 is a schematic view of a scenario of a touch screen webpage of a display device according to some embodiments of the present application through a virtual remote control operation type or a sliding operation type, and as shown in fig. 13, the virtual remote control operation type is taken as an example for illustration, and the virtual remote control may include up, down, left, right, OK determination keys, off, switch, and shutdown function keys, which all establish a mechanism for synchronous control with the display device 200. In response to the focus operation event of the function key, the mobile terminal 300 may capture the focus position, and then, may feed back the focus operation event, such as an up operation event, a shutdown event, and other communication focus positions, to the display device 200, and after the display device 200 receives the focus position and the focus operation event, may synchronously respond to the focus position, and execute the same focus operation event based on the focus position, so as to achieve an effect of consistency between a screen of a screen-cast webpage played by the mobile terminal 300 and a screen-cast screen of the display device 200.

In some embodiments, the user may switch the manner in which the virtual remote control is operated and the type of sliding operation. The control of the screen-cast web page using the virtual remote control operation type may be performed as described above. When the operation type mode of the virtual remote controller is switched to the sliding operation type, that is, when the mobile terminal 300 controls the screen-throwing webpage in the sliding operation type mode, the following manner can be achieved. The display device 200 may first obtain an operation type of controlling the screen-throwing webpage by the user at the mobile terminal 300, obtain a sliding event in the screen-throwing webpage of the mobile terminal 300, a current position coordinate corresponding to the sliding event, and a terminal resolution of the mobile terminal 300 in response to the operation type being a sliding operation type, then convert the terminal resolution and the current position coordinate into a screen resolution and a target position coordinate of the display device 200, and finally generate a sliding instruction according to the sliding event, the screen resolution, and the target position coordinate, and execute the sliding instruction on the screen-throwing webpage.

For example, when the manner of the operation type of the virtual remote controller is switched to the sliding operation type, or when the user controls the screen-casting web page based on the sliding operation type, the mobile terminal 300 may transmit the current position coordinates corresponding to the sliding event in real time, and may feed back the current position coordinates and the sliding event to the display device 200. In order to enable the display device 200 to display the control effect on the screen-throwing webpage on the large screen, a sliding event in the screen-throwing webpage of the mobile terminal 300, a current position coordinate corresponding to the sliding event, and a terminal resolution of the mobile terminal 300 may be acquired, the terminal resolution and the current position coordinate are converted into a screen resolution of the display device 200 and a target position coordinate in the screen-throwing webpage on the large screen, and finally a sliding instruction is generated according to the sliding event, the screen resolution and the target position coordinate, and the sliding instruction is executed on the screen-throwing webpage. Thus, the display device can synchronously display the control effect of the mobile terminal 300 on the screen-throwing webpage.

In some embodiments, the focus operation event includes a break-off screen projection event, and in response to the break-off screen projection event issued by the mobile terminal 300, the display device 200 may control the display 260 to stop displaying the screen projection screen in the user interface, while exiting the web page view control, and destroying the application components.

For example, when the user selects to turn off the screen, the mobile terminal 300 stops the screen, and may generate a turn-off screen-on event or an instruction to turn off the screen, and may simultaneously feed back to the display device 200. After receiving the screen-break event or the screen-break instruction, the display device 200 also stops playing the screen-break screen in the user interface, and exits the webpage view control, such as WebView, and destroys the application component, such as the Activity component. Thus, the consumption of resources in the display device 200 can be reduced after the screen is stopped, and the memory resources are saved.

Fig. 14 is a schematic flow chart of performing input synchronization between a display device and a mobile terminal according to some embodiments of the present application, as shown in fig. 14, in some embodiments, an input field may be disposed in a screen-throwing webpage, and a user may input content that is desired to be searched based on the input field. In response to an input event in the input field, the display apparatus 200 may generate an input prompt instruction for prompting the mobile terminal 300 to display the input field, and transmit the input prompt instruction to the mobile terminal 300. After receiving the input prompt instruction, the mobile terminal 300 may display an input field above the screen-cast web page of the mobile terminal 300, and the user may input the input content desired to be searched in the input field. Thereafter, the display device 200 may receive the input contents input in the input field by the mobile terminal 300 according to the input prompt instruction and control the display 260 to synchronously display the input contents in the input field, and in response to the confirmation event of the input contents, the display device 200 may control the display 260 to display the search results corresponding to the input contents in the user interface.

Illustratively, after the screen-cast web page in the mobile terminal 300 is cast to the display device 200, the display interface in the mobile terminal 300 is the same as the content displayed in the user interface in the display device 200. When the user needs to input the content in the screen-throwing webpage in the large screen of the display device 200, because the input at the large screen is complex, for example, the input method is inconvenient to control, the focus is inconvenient to control, and the like, the display device 200 may generate an input prompt instruction after acquiring the input event in the input field, so that the mobile terminal 300 displays the input field through the input prompt instruction, and the user inputs the input content to be searched in the input field of the mobile terminal 300.

For example, when the display device 200 detects an input event, an input prompt may be generated, where the input prompt is used to prompt the mobile terminal 300 to synchronously display the input fields. Alternatively, when the mobile terminal 300 detects that the large screen end has content input, the mobile terminal 300 may actively pop up an input field for user input. Thereafter, in response to the content input by the user in the input field of the mobile terminal 300, the large screen end of the display device 200 synchronously displays the input content. Meanwhile, after the user input is completed, in response to a confirmation event of the user to the input content at the mobile terminal 300, the display device 200 side may synchronously confirm, perform a search according to the input content, and complete the jump of the search page. Thus, through the synchronous control mechanism between the mobile terminal 300 and the display device 200, when the large screen end is inconvenient to input, the problem of difficult input of the large screen end can be solved through the mobile terminal 300 end to execute the input, and the smoothness of the operation on the screen throwing page is improved. Meanwhile, by inputting contents in the input field and completing the search, the display device 200 may control the display 260 to display a search result corresponding to the input contents in the user interface, or may display an operation event of a previous step in the current interface, for example, after the input field of the large screen loses focus, the mobile terminal 300 may be notified of a remote control operation before displaying.

That is, in some embodiments of the present application, after the display device 200 to be dropped is found and connected at the mobile terminal 300, the mobile terminal 300 may push the drop address to the large screen of the display device 200 by clicking a shortcut bookmark or inputting the drop address, or the display device 200 actively obtains the drop address. The display device 200 may load the screen-throwing address through a web page view control, such as WebView, and at the same time, inform the mobile terminal 300 to control the screen-throwing web page, for example, by controlling the virtual remote controller in a mode of operation type or sliding operation type, etc., and the large screen at the side of the display device 200 synchronously executes the corresponding control event in response to the control of the mobile terminal 300. When there is an input event in the input field of the large screen of the display device 200, the mobile terminal 300 may automatically enter the input interface, perform input in the mobile terminal 300, and synchronously display and respond to the input content in the large screen of the display device 200 through the synchronous control mechanism, thereby completing the process of skipping the input page. In addition, in order to distinguish different types of web page resources, a User Agent identifier such as a User-Agent identifier may be set in the play window. When the display device 200 presents the screen-throwing picture, different webpage resources can be distinguished according to the user agent identification, and different categories of screen-throwing pictures can be presented.

As can be seen from the above technical solutions, the display device 200 provided in the above embodiments responds to a device discovery request sent by the mobile terminal based on the screen-throwing protocol, and extends the screen-throwing protocol based on the device discovery request, so that the screen-throwing protocol supports the screen-throwing event of the web page; responding to a screen throwing request of the screen throwing webpage sent by the mobile terminal, acquiring a screen throwing address input by the mobile terminal, loading the screen throwing address to generate a screen throwing picture of the screen throwing webpage, and controlling a display to display the screen throwing picture in a user interface. The display device can combine the mobile terminal, the screen-throwing webpage and the large screen of the display device together to acquire the screen-throwing address input by the mobile terminal, loads the screen-throwing address through the webpage view control, realizes the effect of watching webpage resources through the large screen, synchronously displays and responds to input contents at the large screen end through the synchronous control mechanism of the display device and the mobile terminal, and further solves the problem that the display device cannot realize the screen throwing of the webpage.

Based on the display device 200, some embodiments of the present application further provide a method for projecting a web page, and the method may be applied to the display device 200 in the foregoing embodiments. Fig. 15 is a flowchart of a method for projecting a web page according to some embodiments of the present application, as shown in fig. 15, in some embodiments, a method for projecting a web page may include the following steps S1-S4:

In some embodiments, when the display device 200 extends the screen-throwing protocol, firstly, the display device 200 may generate a description file, where the description file is used to store the screen-throwing type supported by the display device 200 and the control type for the screen-throwing address, then, a preset field for supporting the screen throwing of the web page is added to the description file, and based on the preset field, the description file is extended to generate an extended description file, and finally, the screen-throwing protocol is extended according to the extended description file, so that the screen-throwing protocol supports the screen throwing of the web page.

For example, the display device 200 may generate the description file after receiving a device discovery request transmitted by the mobile terminal 300 based on a screen-casting protocol. For example, the description file may be description. Xml, which is used to store the type of screen supported by the display device 200 and the type of control over the screen address. After the description file description.xml is obtained, adding a preset field for supporting webpage screen projection, such as a SCPDURL field, into the description file, expanding the description file description.xml based on the SCPDURL field to generate an expanded description file, and finally expanding a screen projection protocol, such as a DLNA protocol, according to the expanded description file, so that the screen projection protocol supports webpage screen projection.

By expanding the screen projection protocol, the screen projection protocol can support the function of webpage screen projection, so that when the mobile terminal 300 performs screen projection with the display device 200 based on the expanded screen projection protocol, the function of webpage screen projection from the mobile terminal 300 to the display device 200 can be realized, for example, the function of webpage screen projection from the mobile phone end to the television end with a large screen can be realized. That is, the application can realize the effect of the screen projection of the webpage by expanding the screen projection protocol, such as the DLNA protocol, and executing the device discovery through the screen projection protocol, thereby solving the problem that the screen projection of the webpage cannot be realized at the display device 200 end.

In order to acquire the screen address input by the mobile terminal 300, in some embodiments, the display device 200 may perform the following procedure. When the display device 200 obtains the screen address input by the mobile terminal 300, it may first receive the screen address transmission instruction sent by the mobile terminal 300, then analyze the screen address transmission instruction, generate the uniform resource identifier carried in the screen address transmission instruction, and then convert the uniform resource identifier into the screen address.

In some embodiments, when the mobile terminal 300 interacts with the display device 200, the application in the mobile terminal 300 may receive a multicast packet at the display device 200 end based on active or passive discovery of a screen-casting protocol, such as DLNA protocol, for example, may be a Nodify multicast packet, or the mobile terminal 300 may send an M-Search multicast packet to the display device 200 by itself. By the mobile terminal 300 transmitting or receiving the multicast packet, the mobile terminal 300 can acquire relevant device information in the display device 200, so that the mobile terminal 300 and the display device 200 can perform interaction based on the relevant device information.

In response to a screen-throwing request for a screen-throwing webpage sent by the mobile terminal 300, the display device 200 may acquire a screen-throwing address input by the mobile terminal 300, that is, after the mobile terminal 300 performs device discovery and selects the display device 200 to be thrown, the mobile terminal 300 may input the screen-throwing address, and the display device 200 may acquire the screen-throwing address input by the mobile terminal 300 and may display content corresponding to the screen-throwing address.

After the display device 200 obtains the screen-throwing address input by the mobile terminal 300, the screen-throwing address can be loaded through the webpage view control, and after loading is completed, a screen-throwing picture corresponding to the screen-throwing webpage is generated. In some embodiments, when the display device 200 generates a screen of a screen-cast webpage, this may be achieved as follows. Firstly, a play window can be created through a webpage view control of an application component, the play window is used for loading a screen throwing address, then a user agent identification is set in the play window, and finally a screen throwing picture of the screen throwing address is generated based on the screen throwing address and the user agent identification.

In some embodiments, the application component may be an Activity component, the web page view control may be WebView, the User Agent identifier may be a User-Agent identifier, and in combination with step S2, the uniform resource identifier may be Uniform Resource Identifier, i.e., a URI, so that after receiving the screen address transmission instruction of the mobile terminal 300, the display device 200 may parse the uniform resource identifier URI carried in the screen address transmission instruction, and then convert the uniform resource identifier URI into a URL, and load the URL address through WebView of the Activity component.

To achieve the effect of consistency of the screen-cast webpage played by the mobile terminal 300 and the screen-cast screen of the display device 200, in some embodiments, the display device 200 may first establish a synchronization control mechanism between the mobile terminal 300 and the display device 200, and then, in response to a user's focus operation event on the screen-cast webpage at the mobile terminal 300, the display device 200 may acquire a focus position input by the user in the screen-cast webpage of the mobile terminal 300 and synchronize the focus position in the screen-cast webpage of the display device 200. And finally, generating a focus operation instruction according to the focus position and the focus operation event, and executing the focus operation instruction on the screen-throwing webpage. The mobile terminal 300 may capture the focal position, and then feed back the focal position and the focal position together to the display device 200, and after the display device 200 receives the focal position and the focal position, the display device 200 may respond to the focal position synchronously, and execute the same focal position based on the focal position, so as to achieve the effect of consistency between the screen of the screen-cast webpage played by the mobile terminal 300 and the screen-cast screen of the display device 200.

According to the technical scheme, the screen projection method of the webpage can combine the mobile terminal, the screen projection webpage and the large screen of the display device together, acquire the screen projection address input by the mobile terminal, load the screen projection address through the webpage view control, realize the effect of watching webpage resources through the large screen, synchronously display and respond to input contents at the large screen end through the synchronous control mechanism of the display device and the mobile terminal, and further solve the problem that the display device cannot realize the screen projection of the webpage.

The same and similar parts of the embodiments in this specification are referred to each other, and are not described herein.

It will be apparent to those skilled in the art that the techniques of embodiments of the present invention may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied essentially or in parts contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method of the embodiments or some parts of the embodiments of the present invention.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A display device, characterized by comprising:

a display configured to display a user interface;

a controller configured to:

2. The display device of claim 1, wherein the controller is further configured to:

sending an operation instruction for controlling the screen throwing webpage to the mobile terminal;

monitoring an operation event of the mobile terminal on the screen-throwing webpage based on the operation instruction;

and generating a webpage control instruction according to the operation event, and executing the webpage control instruction on the screen-throwing webpage.

3. The display device of claim 1, wherein the controller performs the step of expanding the drop protocol based on the device discovery request, further configured to:

Generating a description file, wherein the description file is used for storing a screen throwing type supported by the display equipment and a control type for a screen throwing address;

adding a preset field for supporting webpage screen projection in the description file, expanding the description file based on the preset field, and generating an expanded description file;

and expanding the screen-throwing protocol according to the expansion description file so that the screen-throwing protocol supports webpage screen throwing.

4. The display device of claim 1, wherein the controller performs the step of obtaining a screen address entered by the mobile terminal, and is further configured to:

receiving a screen-throwing address transmission instruction sent by the mobile terminal;

analyzing the screen-throwing address transmission instruction to generate a uniform resource identifier carried in the screen-throwing address transmission instruction;

and converting the uniform resource identifier into a screen throwing address.

5. The display device of claim 1, wherein the controller performs the step of loading the screen-drop address to generate a screen-drop screen of the screen-drop web page, further configured to:

creating a play window through a webpage view control of the application component, wherein the play window is used for loading the screen throwing address;

Setting a user agent identifier in the play window;

and generating a screen-throwing picture of the screen-throwing address based on the screen-throwing address and the user agent identification.

6. The display device of claim 5, wherein the controller is further configured to:

establishing a synchronous control mechanism between the mobile terminal and the display equipment;

responding to a focus operation event of a user on the screen-throwing webpage of the mobile terminal, and acquiring a focus position input by the user in the screen-throwing webpage of the mobile terminal;

synchronizing the focus position in a screen-drop webpage of a display device;

and generating a focus operation instruction according to the focus position and the focus operation event, and executing the focus operation instruction on the screen-throwing webpage.

7. The display device of claim 6, wherein the focus operation event comprises a break-off screen event, the controller further configured to:

in response to the off-screen event, controlling the display to stop displaying the screen in a user interface;

and exiting the webpage view control and destroying the application component.

8. The display device of claim 1, wherein an input field is disposed in the drop-in web page, the controller further configured to:

Responding to an input event of a user in an input field, generating an input prompt instruction, and sending the input prompt instruction to the mobile terminal, wherein the input prompt instruction is used for prompting the mobile terminal to display the input field;

receiving input content input by the mobile terminal in the input field according to the input prompt instruction;

controlling the display to synchronously display the input content in the input field;

and responding to the confirmation event of the input content, and controlling the display to display the search result corresponding to the input content in a user interface.

9. The display device of claim 1, wherein the controller is further configured to:

acquiring an operation type of controlling the screen-throwing webpage by a user at a mobile terminal, wherein the operation type comprises a sliding operation type;

responding to the operation type as the sliding operation type, and acquiring a sliding event in a screen-throwing webpage of the mobile terminal, a current position coordinate corresponding to the sliding event and terminal resolution of the mobile terminal;

converting the terminal resolution and the current position coordinate into the screen resolution and the target position coordinate of display equipment;

And generating a sliding instruction according to the sliding event, the screen resolution and the target position coordinate, and executing the sliding instruction on the screen-throwing webpage.

10. A method for projecting a web page, applied to the display device of any one of claims 1 to 9, comprising: