CN113507638A

CN113507638A - Display device and screen projection method

Info

Publication number: CN113507638A
Application number: CN202110766329.9A
Authority: CN
Inventors: 徐建锋
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2021-07-07
Filing date: 2021-07-07
Publication date: 2021-10-15
Anticipated expiration: 2041-07-07
Also published as: CN113507638B

Abstract

The application discloses a display device and a screen projection method, which comprise the following steps: a display; a controller communicatively coupled to the display, the controller configured to: starting an RTSP (real time streaming protocol) interactive thread in response to receiving a screen projection signal; receiving screen projection data sent by screen projection equipment, wherein the screen projection data comprise a plurality of video frames, and the video frames comprise a plurality of RTP data packets; detecting an RTP data packet, and sending an RTSP message to a screen projection device when the current RTP data packet is discontinuous with the last received data packet so as to enable the screen projection device to reduce the resolution; and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play screen projection data. In the application, the display device informs the screen projection device of reducing the resolution of the screen projection data when the screen is not smoothly played, so that the data volume is reduced, the display device achieves the effect of smooth playing after receiving the screen projection data with low resolution, a user can smoothly watch the screen projection data, and the user experience is improved.

Description

Display device and screen projection method

Technical Field

The application relates to the technical field of display, in particular to a display device and a screen projection method.

Background

The display device is a television product which can realize the bidirectional man-machine interaction function and integrates a plurality of functions such as audio and video, entertainment, data and the like. In order to meet the diversified requirements of users, the display equipment is provided with various applications such as screen projection, audio and video, entertainment and the like, and interacts and exchanges information with the users through a user interface.

For screen projection applications, Miracast screen projection is an important resource sharing mode between display devices, and media resource sharing is realized through RTSP (Real time streaming protocol). For example, when Miracast is used for screen casting, a mobile phone needs to acquire a video frame of media resources to be shared, then compress and encode the video frame, and transmit the video frame to a smart television. And decompressing the received media asset data by the intelligent television, and displaying the decompressed media asset. When part of mobile phones are compressed, an old version compression algorithm is still adopted, so that the data volume sent to the smart television is too large, the smart television cannot process the data in time, the problem of media asset playing delay or screen splash of the smart television is caused, and the user experience effect is poor.

Disclosure of Invention

The application provides a display device and a screen projection method, and aims to solve the technical problem that in the prior art, due to the fact that the data volume of media resources is large after mobile phone compression, an intelligent television cannot process timely, and the user experience effect is poor.

In order to solve the technical problem, the embodiment of the application discloses the following technical scheme:

in a first aspect, an embodiment of the present application discloses a display device, where the display device includes:

a display;

a controller communicatively coupled with the display, the controller configured to:

starting an RTSP (real time streaming protocol) interaction thread in response to the received screen projection signal so as to enable the display equipment and the screen projection equipment to carry out data interaction;

receiving screen projection data sent by the screen projection equipment, wherein the screen projection data comprise a plurality of video frames, and the video frames comprise a plurality of RTP data packets;

-detecting said RTP data packets in a data packet buffer,

when the current RTP data packet is not continuous with the last received RTP data packet, transmitting RTSP information to the screen projection equipment, so that the screen projection equipment reduces the resolution according to the RTSP information; and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play the screen projection data.

In some embodiments, the RTP packet is marked with a packet sequence number, and in the step of sending an RTSP message to the screen projection device when the current RTP packet is not consecutive to the last received RTP packet, the controller is further configured to:

detecting whether the packet sequence number of the current RTP data packet is continuous with the packet sequence number of the last received RTP data packet; and when the packet sequence number is discontinuous, sending an RTSP message to the screen projection equipment.

In some embodiments, prior to said sending an RTSP message to said screen projecting device, said controller is further configured to:

when the packet sequence number is discontinuous, calculating the number of the lost RTP data packets; and when the number of the lost RTP data packets is larger than a preset value, sending RTSP information to the screen projection equipment.

In some embodiments, in the step of calculating the number of lost RTP packets when the packet sequence number is discontinuous, the controller is further configured to:

and subtracting the encapsulation sequence number of the current RTP data packet from the packet sequence number of the last received RTP data packet to obtain the number of the lost RTP data packets.

In some embodiments, after sending an RTSP message to the screen projecting device, the controller is further configured to:

recording the current playing progress of the screen projection data; receiving a feedback message of successful resolution modification sent by the screen projection equipment, and receiving screen projection data which is sent by the screen projection equipment and has reduced resolution; and controlling the display to continue playing the screen projection data according to the reduced resolution ratio based on the playing progress.

a feedback message of successful resolution modification sent by the screen projection equipment is not received; and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen projection data.

In a second aspect, an embodiment of the present application discloses a screen projection method, where the screen projection method includes:

-detecting said RTP data packets in a data packet buffer,

when the current RTP data packet is not continuous with the last received RTP data packet, transmitting RTSP information to the screen projection equipment, so that the screen projection equipment reduces the resolution according to the RTSP information; and controlling a display to play the screen projection data when the current RTP data packet is continuous with the last received RTP data packet.

Compared with the prior art, the beneficial effect of this application is:

when a user sends a screen projection signal to the display device through the screen projection device, the display device responds to the screen projection signal, and the screen projection device and the display device both start RTSP (real time streaming protocol) interaction threads. And the screen projection equipment compresses and packages the screen projection data into a plurality of RTP data packets and sends the RTP data packets to the display equipment. The display equipment receives and detects the RTP data packet sent by the screen projection equipment, and judges whether the current RTP data packet is continuous with the last received RTP data packet. If the RTSP is continuous, the display device displays screen projection data on the display, and if the RTSP is discontinuous, the display device sends an RTSP message to the screen projection device to request the screen projection device to reduce the resolution. In the application, the display device enables the screen projection device to reduce the resolution ratio of screen projection data playing by sending the RTSP message used for indicating the screen projection device to reduce the resolution ratio, so that the data volume is reduced, the display device achieves the smooth playing effect after receiving the screen projection data sent by the screen projection device and with the reduced resolution ratio, a user can watch the screen projection data smoothly, and the user experience is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic diagram illustrating an operational scenario between a display device and a control apparatus according to some embodiments;

a block diagram of the hardware configuration of the control device 100 according to some embodiments is illustrated in fig. 2;

a block diagram of a hardware configuration of a display device 200 according to some embodiments is illustrated in fig. 3;

a schematic diagram of a software configuration in a display device 200 according to some embodiments is illustrated in fig. 4;

a flow diagram of a screen projection method according to some embodiments is illustrated in fig. 5;

another flow diagram of a screen projection method according to some embodiments is illustrated in fig. 6;

a timing diagram of a method of screen projection according to some embodiments is illustrated in fig. 7.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may input a user instruction through a key on a remote controller, voice input, control panel input, etc., to control the display apparatus 200.

In some embodiments, the smart device 300 (e.g., mobile terminal, tablet, computer, laptop, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received by a module configured inside the display device 200 to obtain a voice command, or may be received by a voice control device provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

Fig. 3 shows a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In some embodiments the controller comprises a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, performing display of video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the external control apparatus 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals for controlling the apparatus 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

And the CPU is used for executing the operating system and the application program instructions stored in the memory and executing various application programs, data and contents according to various interaction instructions for receiving external input so as to finally display and play various audio and video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on the direct display device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, a system of a display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together make up the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (windows) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage the icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

The hardware or software architecture in some embodiments may be based on the description in the above embodiments, and in some embodiments may be based on other hardware or software architectures that are similar to the above embodiments, and it is sufficient to implement the technical solution of the present application.

Based on the display device 200, a user can download a screen projection application in an application program center of the display device 200, the screen projection application can be an application program based on a Miracast (wireless display) function, the Miracast transmits screen projection data of the screen projection device to the display device 200 by using a WIFI Point-to-Point (Point 2Point) network transmission function to play, and therefore the purpose of resource sharing is achieved, wherein the screen projection data can be audio and video data, and the screen projection device can be a terminal device such as a smart phone, a tablet computer, a computer and the like.

When the Miracast realizes the screen projection function, the media resource sharing needs to be realized by utilizing a WFD (Wi-Fi Display) interaction protocol. The WFD interaction protocol utilizes rtsp (real Time Streaming protocol) protocol to implement capability negotiation between the screen projection device and the display device. According to the RTSP, the screen projection device and the display device and the stages M3 and M4, some capability parameters needing to be negotiated at two ends of the device are specified, wherein one is video capability negotiation, and the two ends negotiate the resolution, refresh rate, compression algorithm and the like of screen projection data to be played. The display device 200 generally desires that the screen projection device employ a higher compression algorithm to reduce the amount of data as much as possible by compression, but since some screen projection devices are older or lower end, an older version of the compression algorithm, such as the h.264level 3.2 or earlier compression algorithm, is still used. Thus, the screen projection data volume sent from the screen projection device is very large, which causes the display device 200 to be out of time to process, and the problem of screen splash caused by media asset delay or packet loss occurs. In order to solve the above problems, the present application provides a display device and a screen projection method in some embodiments. It should be noted that, the display device in this application may refer to not only the smart television, but also a computer, a tablet computer, and the like.

The screen projection process provided by the embodiment of the application is described below with reference to the accompanying drawings.

A flow diagram of a screen projection method according to some embodiments is illustrated in fig. 5. The present application provides, in some embodiments, a display device 200, the display device 200 including a display 260 and a controller 250, the controller 250 communicatively coupled to the display 260, the controller configured to perform the screen projection process of fig. 5. With reference to fig. 5, the screen projection process is as follows:

s501: and responding to the received screen projection signal, starting an RTSP interaction thread so as to enable the display equipment and the screen projection equipment to carry out data interaction.

In some embodiments, a user may input a screen projection signal at a screen projection device, wherein the screen projection device is provided with a screen projection application, and the screen projection application is based on a Miracast screen projection function. For example, a screen projection control is arranged on the screen projection device, and a user clicks or selects the screen projection control through touch control to start a Miracast screen projection function. After the Miracast screen projection function is started, the screen projection device sends a screen projection signal to the display device 200, the display device 200 responds to the screen projection signal, starts the Miracast screen projection function of the display device, and enables the display device and the screen projection device to perform data interaction based on the Miracast.

In some embodiments, when the display device 200 starts the Miracast screen projection function, it starts its own P2P network. Of course, when the Miracast screen projection function is started, the screen projection device also starts its own P2P network. Based on the P2P network, the display device 200 establishes a connection with the screen projection device, and the two ends interact with each other through the network part to obtain the IP addresses of each other.

In some embodiments, after the P2P network connection is established, the display device 200 and the screen projection device respectively start an RTSP interaction thread to perform a handshake interaction phase, so as to implement data interaction between the display device 200 and the screen projection device.

S502: and receiving screen projection data sent by the screen projection equipment, wherein the screen projection data comprise a plurality of video frames, and the video frames comprise a plurality of RTP data packets.

In some embodiments, in the RTSP interaction thread of the display device 200 and the screen projection device, the screen projection device 200 parses the screen projection data into a plurality of video frames, each video frame is composed of a plurality of RTP packets, each RTP packet is marked with a sequcnumber (packet sequence number), and the packet sequence number indicates the sequence continuity of the plurality of RTP packets. The screen projection device sends the plurality of RTP data packets to the display device 200, and the display device 200 receives the plurality of RTP data packets.

S503: and detecting the RTP data packet.

In some embodiments, the display device 200 may turn on the gatekeeper thread upon receiving the screen projection data. And detecting the RTP data packet through a packet guarding thread. Here, the packet guarding procedure mainly detects the continuity of the RTP packets.

S504: and when the current RTP data packet is continuous with the last received RTP data packet, controlling the display to play the screen projection data.

In some embodiments, if the display device 200 can process the received screen projection data in time, that is, decode the screen projection data in time, and the packet watching process can detect that the RTP data packets are in a continuous state all the time, the display device 200 removes the packet header from the RTP data packets, and sends the RTP data packets to the player for playing and displaying on the display.

S505: and when the current RTP data packet is not continuous with the last received RTP data packet, sending an RTSP message to the screen projection equipment, so that the screen projection equipment reduces the resolution according to the RTSP message.

In some embodiments, when detecting the current RTP packet, the packet watching thread finds that the current RTP packet is not continuous with the last received RTP packet, that is, when detecting that the RTP packet is lost, the display device 200 sends an RTSP message to the screen projection device. And after receiving the RTSP message, the screen projection equipment reduces the resolution of the screen projection data and performs the packaging of the screen projection data again.

In some embodiments, in the RTSP interaction thread of the display device 200 and the screen projection device, after the display device 200 detects the RTP packet loss, a new RTSP message is added, which modifies the resolution through SET _ PARAMETER.

For example, in the RTSP interaction thread at the beginning of the display device 200, during the video stream negotiation process in the M3/M4 stage, the screen projection device tells the display device 200 the following video negotiation parameters:

wfd _ video _ formats 0001010200000080000000000000000000000000000 none. According to Wi-Fi _ Display _ Specification section6.1.3 (section 6.1.3 of the Wi-Fi standard protocol Specification), the RTSP message 0080 indicates playback at 1920 × 1080P30 (resolution 1920 × 1080, refresh rate 30 fps).

After the display device 200 detects the RTP packet loss, it sends an RTSP message M17 to the screen projection device, where the message content is as follows:

wfd _ video _ formats 0001010200000020000000000000000000000000000 none. 0020 in this RTSP message indicates play at 1280 × 720P30 (resolution 1280 × 720, refresh rate 30 fps).

In some embodiments, based on the packet sequence number marked in the RTP packet, the display apparatus 200 detects whether the packet sequence number of the current RTP packet is consecutive to the packet sequence number of the last received RTP packet. And when the packet sequence number is discontinuous, sending an RTSP message to the screen projection equipment.

In some embodiments, according to previous playing experience, if the number of RTP packets lost is within a certain range, the fluency of the entire screen-projected data playing is not substantially affected. Thus, prior to said sending an RTSP message to said screen projecting device, said controller 250 is further configured to: and when the packet sequence number is discontinuous, calculating the number of the lost RTP data packets. And when the number of the lost RTP data packets is larger than a preset value, sending RTSP information to the screen projection equipment. Here, the controller 250 may set a preset value to 5, and a specific value of the preset value may be set by a worker according to past experience.

In some embodiments, when detecting that the packet sequence number of the current RTP packet is not consecutive to the packet sequence number of the last received RTP packet, the packet guarding thread in the display device 200 may further calculate the number of the lost RTP packets, and obtain the number of the lost RTP packets by subtracting the packet sequence number of the current RTP packet from the packet sequence number of the last received RTP packet.

For example, the packet guarding thread in the display device 200 detects that the packet sequence number of the current RTP packet is 6, the packet sequence number of the last received RTP packet is 3, the two packet sequence numbers are not consecutive, and the number of the currently lost RTP packets is 3 through 6-3. And 3 is less than the preset value 5, the RTSP message may not be sent first. If the packet sequence number of the current RTP data packet is detected to be 9, the packet sequence number of the last received RTP data packet is 3, the two packet sequence numbers are not continuous, and the difference between the two packet sequence numbers is 6. If the number of lost RTP packets exceeds the preset value of 5, the display device 200 sends an RTSP message to the screen projection device.

After the display device 200 transmits the RTSP message, the display device 200 may further determine whether the screen projection device has modified the resolution according to the message. Another flow diagram of a screen projection method according to some embodiments is illustrated in fig. 6. In conjunction with fig. 6, the controller 250 is further configured to perform the following processes:

s601: and recording the current playing progress of the screen projection data.

In some embodiments, after the display device 200 transmits the RTSP message, the display device 200 records the current play progress of the screen shot data. For example, the display device 200 records that the current screen projection data is played for 48%, and for another example, the display device 200 records that the current screen projection data is played for 35 seconds at 20 minutes.

S602: and judging whether a feedback message of successfully modifying the resolution ratio is received.

In some embodiments, after the screen projection device receives the RTSP message, the playback resolution of the projected data is reduced based on the indication in the RTSP message. After the screen projection device successfully modifies the resolution, the screen projection device sends a feedback message that the modification is successful to the display device 200, so that the display device 200 knows that the resolution modification by the screen projection device is successful. Of course, if the screen projection device is not successfully modified, the feedback message of successfully modifying the resolution ratio is not sent.

S603: and receiving screen projection data which is sent by the screen projection equipment and has reduced resolution.

In some embodiments, after the screen projection device reduces the playing resolution of the screen projection data, the screen projection data is packaged by a new package, the screen projection data with the reduced resolution is packaged into a plurality of RTP data packets, and the newly packaged screen projection data is sent to the display device 200. The display device 200 receives the low-resolution screen projection data transmitted by the screen projection device.

S604: and controlling the display to continue playing the screen projection data according to the reduced resolution ratio based on the playing progress.

In some embodiments, after receiving the screen projection data after the resolution is reduced, the display device 200 searches for a playing progress of the previously recorded screen projection data, and continues playing the screen projection data based on the recorded playing progress.

S605: and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen projection data.

In some embodiments, the display device 200 does not receive the feedback message that the resolution modification is successful, that is, the screen projection device does not perform the operation of reducing the resolution, and the user has a delay or a screen splash when viewing the screen projection data, which results in poor viewing experience, at this time, the display device 200 may stop the RTSP interaction thread, exit the Miracast screen function, that is, exit the playing process of the screen projection data.

The process of dropping the screen is further described below with reference to the drawings.

A timing diagram of a method of screen projection according to some embodiments is illustrated in fig. 7. As shown in fig. 7, a user triggers a screen projection function through a screen projection application installed on the display device 200, after the display device 200 starts the Miracast screen projection function, a screen projection signal is sent to the screen projection device, the display device 200 and the screen projection device both start respective P2P networks, and a P2P network connection is established between the two. Subsequently, the display device 200 and the screen projection device both start their RTSP interaction threads. The screen projection data packet of the screen projection device 200 is an RTP data packet and is sent to the display device 200, and the display device 200 detects the received RTP data packet. When the number of RTP packets lost is greater than the preset value, the display device 200 sends an RTSP message to the screen projection device. And the screen projection equipment receives the RTSP message and reduces the resolution of the screen projection data according to the RTSP message. The screen projection equipment feeds back the message of successful resolution modification to the display equipment 200, and feeds back the screen projection data with reduced resolution to the display equipment 200. the display equipment 200 continues to play the screen projection data with reduced resolution based on the previous playing progress so as to realize smooth playing of the screen projection data.

In the application, the display device enables the screen projection device to reduce the resolution ratio of screen projection data playing by sending the RTSP message used for indicating the screen projection device to reduce the resolution ratio, so that the data volume is reduced, the display device achieves the smooth playing effect after receiving the screen projection data sent by the screen projection device and with the reduced resolution ratio, a user can watch the screen projection data smoothly, and the user experience is improved.

Based on the same inventive concept as the display device, the embodiment of the application further provides a screen projection method, which comprises the following steps: the display device 200 starts an RTSP interaction thread in response to receiving the screen projection signal, so that the display device 200 performs data interaction with the screen projection device. The display device 200 receives screen projection data sent by the screen projection device, wherein the screen projection data comprises a plurality of video frames, and the video frames comprise a plurality of RTP data packets. After receiving screen projection data, the display device 200 needs to detect the RTP data packet, and when a current RTP data packet is not continuous with a last received RTP data packet, sends an RTSP message to the screen projection device, so that the screen projection device reduces a resolution according to the RTSP message; and controlling a display to play the screen projection data when the current RTP data packet is continuous with the last received RTP data packet.

In some embodiments, the RTP packets are marked with packet sequence numbers, and the display apparatus 200 detects whether the packet sequence number of the current RTP packet is consecutive to the packet sequence number of the last received RTP packet. And when the packet sequence number is not continuous, the display device 200 sends an RTSP message to the screen projection device.

In some embodiments, the display apparatus 200 further calculates the number of lost RTP packets when the packet sequence number is not consecutive. And when the number of the lost RTP data packets reaches a preset value, the display device 200 sends RTSP information to the screen projection device.

Since the above embodiments are all described by referring to and combining with other embodiments, the same portions are provided between different embodiments, and the same and similar portions between the various embodiments in this specification may be referred to each other. And will not be described in detail herein.

It is noted that, in this specification, relational terms such as "first" and "second," and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a circuit structure, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such circuit structure, article, or apparatus. Without further limitation, the presence of an element identified by the phrase "comprising an … …" does not exclude the presence of other like elements in a circuit structure, article, or device comprising the element.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

The above embodiments of the present application do not limit the scope of the present application.

Claims

1. A display device, characterized in that the display device comprises:

a display;

-detecting said RTP data packets in a data packet buffer,

2. The display device of claim 1, wherein the RTP packet is marked with a packet sequence number, and in the step of sending an RTSP message to the screen-casting device when the current RTP packet is not consecutive with a last received RTP packet, the controller is further configured to:

detecting whether the packet sequence number of the current RTP data packet is continuous with the packet sequence number of the last received RTP data packet;

and when the packet sequence number is discontinuous, sending an RTSP message to the screen projection equipment.

3. The display device of claim 2, wherein prior to said sending an RTSP message to the screen-casting device, the controller is further configured to:

when the packet sequence number is discontinuous, calculating the number of the lost RTP data packets;

and when the number of the lost RTP data packets is larger than a preset value, sending RTSP information to the screen projection equipment.

4. The display device according to claim 3, wherein in the step of calculating the number of lost RTP packets when the packet sequence number is discontinuous, the controller is further configured to:

5. The display device of claim 1, wherein after sending the RTSP message to the screen-casting device, the controller is further configured to:

recording the current playing progress of the screen projection data;

receiving a feedback message of successful resolution modification sent by the screen projection equipment, and receiving screen projection data which is sent by the screen projection equipment and has reduced resolution;

and controlling the display to continue playing the screen projection data according to the reduced resolution ratio based on the playing progress.

6. The display device of claim 1, wherein after sending the RTSP message to the screen-casting device, the controller is further configured to:

a feedback message of successful resolution modification sent by the screen projection equipment is not received;

and stopping the RTSP interaction thread, and controlling the display to exit the playing process of the screen projection data.

7. The display device of claim 3, wherein the controller is further configured to:

and setting the preset value to be 5.

8. A screen projection method is characterized by comprising the following steps:

-detecting said RTP data packets in a data packet buffer,

9. The screen shot method of claim 8, wherein the RTP packet is marked with a packet sequence number, and when the current RTP packet is not consecutive to the last received RTP packet, the sending an RTSP message to the screen shot device comprises:

10. The screen projection method of claim 9, wherein when the packet sequence number is not consecutive, sending an RTSP message to the screen projection device comprises:

and when the number of the lost RTP data packets reaches a preset value, sending RTSP information to the screen projection equipment.