WO2020073777A1 - Media processing method and apparatus - Google Patents

Media processing method and apparatus Download PDF

Info

Publication number
WO2020073777A1
WO2020073777A1 PCT/CN2019/106327 CN2019106327W WO2020073777A1 WO 2020073777 A1 WO2020073777 A1 WO 2020073777A1 CN 2019106327 W CN2019106327 W CN 2019106327W WO 2020073777 A1 WO2020073777 A1 WO 2020073777A1
Authority
WO
WIPO (PCT)
Prior art keywords
terminal device
data
media
server
streaming
Prior art date
Application number
PCT/CN2019/106327
Other languages
French (fr)
Chinese (zh)
Inventor
靳丽
吴亮
折小强
杜衡
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020073777A1 publication Critical patent/WO2020073777A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H04N21/4385—Multiplex stream processing, e.g. multiplex stream decrypting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4627—Rights management associated to the content

Definitions

  • the embodiments of the present application relate to the communication field, and in particular, to a media processing method and device.
  • Cloud video has become one of the mainstream of the video industry.
  • Cloud video can be implemented based on cloud technology.
  • Cloud video based on cloud technology can load video pages through the browser of the cloud server to obtain video images.
  • Cloud The server then streams the video picture and transmits it to the terminal device, and the terminal device plays the video picture.
  • the cloud server performs video decoding on each frame of the rendered video, and then merges the decoded rendered image with the page information, and re-encodes the merged video information with the page information to output a support terminal.
  • the encoded video stream played by the device is sent to the terminal device for playback.
  • the cloud server decodes every frame of the video frame rendering, which consumes a lot of resources on the cloud server, and the cloud server recodes the video stream that combines the decoded video information and page information to increase the broadcast control delay. Time, thereby affecting the user experience.
  • An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
  • a first aspect of an embodiment of the present application provides a media processing method, including:
  • the server obtains the media playback request from the terminal device; the server obtains the streaming data of the media playback page according to the media playback request, and the media information and streaming media data of the media requested by the terminal device to play, where Is the configuration information of the streaming media data; the server sends the streaming data and the media information to the terminal device, and the streaming data is used by the terminal device to display the media on the media playback page
  • the information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display the media playback page; the server sends the streaming media data to the terminal device to enable the terminal
  • the device uses the media player to play the streaming media data on the media playing page.
  • the server sends the streaming data of the media playback page and the streaming media data of the media requested by the terminal device to the terminal device, and the terminal device displays the media playback page and plays the streaming media data, so that the server does not need to Decode and merge the streaming data and streaming media data to reduce server resource consumption and broadcast control delay.
  • the server obtains streaming data of a media playback page according to the media playback request, and media requested by the terminal device to play
  • the media information and streaming media data include: the server loads the media playback page corresponding to the media playback request, and streams the media playback page to obtain streaming data of the media playback page;
  • the media playback page obtains the media information and the streaming media data.
  • the streaming data, the media information, and the streaming media data can be obtained to provide a
  • the specific method of acquiring the streaming data, the media information, and the streaming media data improves the feasibility of the solution.
  • the streaming media data includes an audio data stream and a video data stream.
  • the server sending the streaming media data to the terminal device includes the server sending the audio data stream and the video data stream to the terminal device, respectively. It can be seen from the second implementation manner of the first aspect that the audio data stream and the video data stream of the streaming media data are separately sent to the terminal device, which improves the reliability of the audio data stream and the video data stream.
  • the method further includes: the server acquiring decoder information of the terminal device from the terminal device, the decoder information including decoding supported by the terminal device Format; the server sending the streaming media data to the terminal device includes the server sending the streaming media data to the terminal device in an encoding format of a target encoding format, the target encoding format and the decoder Information matching. It can be seen from the third implementation manner of the first aspect that the server sends the streaming media data in the encoding format of the target encoding format to the terminal device, so that the terminal device can decode and play the streaming media data using a matching decoder.
  • the streaming media data is an encrypted stream
  • the method further includes the server sending a digital rights management DRM certificate to the terminal device, where the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
  • the method further includes: the server obtains a connection request from the terminal device, the connection request includes device information of the terminal device; the server Triggering a device management platform to verify the terminal device according to the device information; when the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device. It can be seen from the fifth implementation of the first aspect that by triggering the device management platform to verify the terminal device, only the terminal device that passes the verification will maintain the connection relationship with the server, which improves security.
  • the method further includes: the server caches the streaming media data in a first cache area; when the streaming media is cached in the first cache area When the data amount of the data reaches the first preset threshold, the server sends a reception instruction to the terminal device, where the reception instruction is used to instruct the terminal device to receive the streaming media data. It can be seen from the sixth implementation manner of the first aspect that the streaming media data is cached in the first buffer area, which improves the integrity and fluency of the streaming media data.
  • the server sending the streaming media data to the terminal device includes: the server receiving a data acquisition request sent by the terminal device, the data acquisition request includes a data type of the requested data;
  • the terminal device sends the buffered streaming media data. It can be seen from the seventh implementation of the first aspect that the server sends streaming media data to the terminal device according to the data type of the data request, which improves the accuracy of sending streaming media data.
  • a second aspect of an embodiment of the present application provides a media processing method, including:
  • the terminal device When the terminal device obtains the media playback request input by the user, the terminal device sends a media playback request to the server; the terminal device receives the media information sent by the server and the streaming data of the media playback page, and the media information is Configuration information of the streaming media data; the terminal device creates a media player based on the media information; the terminal device displays the media playback page based on the streaming data; the terminal device receives streaming media data sent by the server , The terminal device uses the media player to play the streaming media data on the media playing page. It can be seen from the second aspect that the terminal device separately decodes and displays the streaming data of the media playback page and decodes and plays the streaming media data, thereby reducing the broadcast control delay.
  • the streaming media data includes an audio data stream and a video data stream
  • the terminal device uses the media playback page
  • the playing of the streaming media data by the media player includes: the terminal device using the media player to play the audio data stream and the video data stream on the media playing page; the terminal device according to the audio and video
  • the frame display time PTS synchronizes the audio data stream and the video data. It can be seen from the first implementation of the second aspect that the terminal device synchronizes the audio data stream and the video data according to the PTS, so that the audio data stream and the video data can be played synchronously in the media player .
  • the terminal device uses the media playback on the media playback page Before the player plays the streaming media data, the method further includes: the terminal device receives a digital rights management DRM certificate sent by the server; and the terminal device decrypts the streaming media data according to the DRM certificate. It can be seen from the second implementation manner of the second aspect that the terminal device decrypts the streaming media data by using a DRM certificate, thereby improving the security of the streaming media data.
  • the terminal Before the device receives the streaming media data sent by the server, the method further includes: the terminal device receives a receiving instruction sent by the server; the terminal device sends a data acquisition request to the server according to the receiving instruction
  • the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send streaming media data corresponding to the data type.
  • the terminal After the device receives the streaming media data sent by the server, the method further includes: the terminal device caches the streaming media data in a second cache area; when the streaming media data is cached in the second cache area When the amount of data reaches the second preset threshold, the terminal device stops receiving the streaming media data. It can be seen from the fourth implementation of the second aspect that the terminal device caches the streaming media in the second buffer area, which improves the integrity and fluency of the streaming media data.
  • the method further includes: the terminal device sends decoder information of the terminal device to the server, and the decoder information is used to instruct the server to send the encoding format as The streaming media data in a target encoding format, the target encoding format matches the decoder information. It can be seen from the fifth implementation manner of the second aspect that the terminal device sends decoder information to the server so that the encoding format of the requested data matches the decoder information of the terminal device.
  • the method further includes: the terminal device sends a connection request to the server, and the connection request includes device information of the terminal device.
  • the device information is used by the server to verify the terminal device. It can be seen from the sixth implementation manner of the second aspect that the terminal device sends device information to the server, so that the server can trigger verification of the terminal device based on the device information, thereby improving security.
  • all The streaming media data includes the audio data stream and the video data stream
  • the terminal device buffering the audio and video data in the second buffer area includes: the terminal device buffering the audio data stream and the video data, respectively Flow in the second buffer area.
  • all The terminal device using the media player to play the streaming media data on the media playback page includes: the terminal device reading the first data from the second cache area and caching the first data In the third cache area of the media player; the terminal device reads the second data from the third cache area and uses the media player to play the second data on the media playback page data. It can be seen from the eighth implementation of the second aspect that the terminal device caches the first data in the third cache area of the media player, so that the media player can directly read the second data from the third cache area for playback, Improve the integrity and fluency of streaming media data.
  • a third aspect of an embodiment of the present application provides a server.
  • the server includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one processor Connected by line;
  • the at least one processor calls the instruction to perform the message processing or control operation performed on the server side in the first aspect.
  • a fourth aspect of an embodiment of the present application provides a terminal device.
  • the terminal device includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one The processor is connected by line;
  • the at least one processor calls the instruction to perform the message processing or control operation performed on the terminal device side in the second aspect.
  • a fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the first aspect or any possible implementation manner of the first aspect .
  • a sixth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect .
  • a seventh aspect of the embodiments of the present application provides a computer program product containing instructions, characterized in that, when it is run on a computer, the computer is caused to execute the method of the first aspect or any possible implementation manner of the first aspect.
  • An eighth aspect of an embodiment of the present application provides a computer program product containing instructions, which, when it runs on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect.
  • the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device according to the streaming data Display the media playback page, and decode and play the streaming media data by the terminal device. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of an embodiment of a media processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of another embodiment of a media processing method provided by an embodiment of this application.
  • FIG. 5 is a schematic diagram of a streaming media data cache provided by an embodiment of this application.
  • FIG. 6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application.
  • FIG. 7 is a schematic block diagram of a server provided by an embodiment of this application.
  • FIG. 8 is a schematic block diagram of a terminal device provided by an embodiment of this application.
  • FIG. 9 is another schematic block diagram of a server provided by an embodiment of this application.
  • FIG. 10 is another schematic block diagram of a terminal device provided by an embodiment of this application.
  • FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application.
  • An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the schematic diagram of the application scenario may include a server 101 and a terminal device 102.
  • the server 101 may serve as a media delivery party, and the terminal device 102 may serve as a media delivery party.
  • the terminal device 102 sends a media playback request to the server 101.
  • the server 101 sends the streaming data of the media playback page and the media information and streaming media data of the media requested by the terminal device 102 to the terminal device 102 according to the media playback request, and then the terminal
  • the device 102 displays a media playback page, and creates a media player according to the media information, and causes the media player to play streaming media data.
  • the terminal device 102 may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem.
  • the terminal device 102 may be a set-top box, a smart TV, a mobile station (MS), a subscriber unit (subscriber unit), a cellular phone (cellular phone), a smart phone (smart phone), a wireless data card, and a personal digital assistant ( Personal digital assistant (PDA) computer, tablet computer, wireless modem (modem), handheld device (handset), laptop computer (laptop computer), machine type communication (machine type communication (MTC) terminal, etc.
  • the server 101 provided in the embodiment of the present application may be a cloud server, and the cloud server has the characteristics of being highly distributed and highly virtualized.
  • the cloud server may send the streaming data and streaming media data of the media playback page obtained by the media playback request to the terminal device 102 in a remote delivery mode.
  • the media playback request may be a Web video broadcast request, where the video requested by the Web video broadcast request is Web video. That is to say, the cloud server can send the streaming data corresponding to the video playback page of the Web video broadcast request and the streaming media data of the Web video to the terminal device 102 by remote drop.
  • FIG. 2 is a system framework diagram provided by an embodiment of the present application.
  • the system framework diagram includes a server and a terminal device.
  • the server may serve as a media delivery party
  • the server may include a browser 201 and a delivery module 202
  • the terminal device may serve as a media recipient
  • the terminal device includes a delivery receiving module 203 and a media player 204.
  • the browser 201 may be used to load a media playback page corresponding to the media playback request and detect media information in the media playback page.
  • the browser 201 loads the media playback page corresponding to the media broadcast request, and can obtain page element images, and media information and streaming media data of the media requested to be played by the receiving module, where the media information is configuration information of the streaming media data.
  • the browser 201 outputs page element images, media information, and streaming media data to the delivery module 202.
  • the delivery module 202 may be used to receive and process the interactive instruction of the delivery receiving module 203, and deliver media information, streaming data and media data of the media playback page to the delivery receiving module 203.
  • the delivery module 202 has pixel processing capability and streaming media processing capability.
  • the pixel processing capability is to stream the page element image obtained from the browser 201 to obtain streaming data of the media playback page
  • the streaming media processing capability is to use the streaming media protocol Obtain the encapsulated streaming media data from the browser 201, and decapsulate the streaming media protocol, and cache the decapsulated streaming media data in the cache area.
  • the delivery receiving module 203 may be used for interaction with the delivery module 202, as well as receiving media information, streaming data and streaming media data of media playback pages.
  • the media player 204 can use the local multimedia processing capabilities to decode the streaming media data, and synchronize the decoded audio and video information and other processes to restore the playback process performed on the server, thereby achieving media playback.
  • a media processing method provided by an embodiment of the present application may include the following steps:
  • the terminal device sends a connection request to the server.
  • the terminal device Before playing the media, the terminal device initiates a connection request to the server, and the connection request carries the device information of the terminal device.
  • the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device.
  • the device information may be as shown in Table 3-1.
  • the server triggers the device management platform to verify the terminal device.
  • the server acquires the device information of the terminal device from the connection request sent by the terminal device, and the server triggers the device management platform to verify the terminal device according to the device information. It should be noted that the server can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
  • the server When the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device, allowing the terminal device to access the server.
  • the server may directly allow the terminal device to access the server without verifying the terminal device.
  • the terminal device sends decoder information to the server.
  • the terminal device After the terminal device is connected to the server, the terminal device uploads the decoder information of the terminal device to the server, where the decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
  • the server initializes the system.
  • the server After the server obtains the decoder information from the terminal device, the server confirms the decoding capability of the terminal device and initializes the system according to the decoder information.
  • the terminal device sends a media playback request.
  • the terminal device sends a media playback request to the server, where the media playback request includes key instruction information input by the user.
  • the key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices.
  • the key instruction information may be as shown in Table 3-3.
  • the server obtains streaming data, media information, and streaming media data.
  • the server determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the server loads the media playback page and streams the media playback page to Obtain the streaming data of the media playback page, and obtain the media information and streaming media data of the media requested to be played by the terminal device from the media playback page.
  • the streaming media data is the media data of the media requested by the media playback request
  • the streaming data is the page data of the media playback page corresponding to the media playback request.
  • the streaming data can be a media playback operation button, a fast forward icon, and a fast backward Information such as icons or pause icons. It should be noted that the streaming data may be a picture or a video stream.
  • the media information includes configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data.
  • the server caches the streaming media data in the first cache area.
  • the server caches the streaming media data acquired from the media playback page in the first cache area, and monitors the data volume of the streaming media data cached in the first cache area.
  • the server caches the audio data stream and the video data stream of the streaming media data in the first buffer area, respectively.
  • the server sends streaming data and media information.
  • the server sends the streaming data and media information of the media playback page to the terminal device, where the media information is used by the terminal device to create a media player.
  • the terminal device creates a media player and displays a media playback page.
  • the terminal device obtains the media information from the server, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data.
  • the media player plays the audio and video information corresponding to the streaming media data.
  • the terminal device obtains the streaming data of the media playback page from the server, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
  • the media player created by the terminal device may be embedded in the media playback page.
  • the server sends a reception instruction to the terminal device.
  • the server sends a receiving instruction to the terminal device, and the receiving instruction is used to instruct the terminal device to receive the streaming media data.
  • the terminal device sends a data acquisition request to the server.
  • the terminal device After acquiring the receiving instruction from the server, the terminal device sends a data acquisition request to the server, where the data acquisition request includes the data type of the requested data.
  • the data type may be as shown in Table 3-4.
  • the server sends streaming media data to the terminal device.
  • the server acquires the data acquisition request sent by the terminal device, and determines the data type of the data requested by the terminal device from the data acquisition request.
  • the server sends streaming media data whose encoding format is the target encoding format to the terminal device according to the decoder information of the terminal device, where the target encoding format matches the decoder information.
  • the server when the streaming media data is an ES stream, the server separately sends the audio data stream and the video data stream of the streaming media data to the terminal device.
  • the server may also send a digital rights management (digital rights management (DRM) certificate to the terminal device.
  • DRM digital rights management
  • the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data .
  • the terminal device caches the streaming media data in the second cache area.
  • the terminal device receives streaming media data from the server, and caches the streaming media data in the second buffer area of the terminal device. During the process of receiving the streaming media data, the terminal device can monitor the data volume of the streaming media data cached in the second cache area, and when the data volume of the streaming media data cached in the second cache area reaches the second pre- When the threshold is set, the terminal device stops requesting streaming media data.
  • the terminal device may resume receiving streaming media data.
  • the terminal device when the streaming media data is an ES stream, the terminal device separately caches the audio data stream and the video data stream of the streaming media data in the second buffer area.
  • the terminal device caches the streaming media data in the third buffer area.
  • the terminal device When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the terminal device reads the first data from the second buffer area, and buffers the read first data in the media player The third cache area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area. In the process of reading the first data, the terminal device monitors the data amount of the first data buffered in the third buffer area, and when the data amount of the first data in the third buffer area reaches the third preset threshold, the terminal The device stops reading the first data.
  • the terminal device when the streaming media data is an ES stream, after the terminal device reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area, the terminal device The presentation time (PTS) of the frame synchronizes the audio data stream and the video data stream, and buffers the synchronized audio data stream and video data stream in the third buffer area.
  • PTS presentation time
  • the streaming media data is an ES stream
  • the first data includes an audio data stream and a video data stream
  • the terminal device reads the audio data stream and the video data stream from the second buffer area, Cache in the third cache area respectively.
  • the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
  • the terminal device uses a media player to play streaming media data.
  • the terminal device reads the second data from the third buffer area, where the second data is streaming media data.
  • the terminal device uses a media player to decode and play the streaming media data.
  • the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
  • the terminal device when the streaming media data is an encrypted stream, before the terminal device decodes and plays the second data, the terminal device may also obtain a DRM certificate sent by the server and use the DRM certificate to perform encryption on the streaming media data Decrypt.
  • the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device displays according to the streaming data Media playback page, and the terminal device decodes and plays streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
  • another media processing method provided by an embodiment of the present application may include the following steps:
  • the delivery receiving module sends a connection request to the delivery module.
  • the drop receiving module Before playing the media, the drop receiving module initiates a connection request to the drop module, the connection request carrying device information of the terminal device.
  • the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device.
  • the device information may be as shown in Table 3-1.
  • the drop module triggers the device management platform to verify the terminal device.
  • the drop module obtains the device information of the terminal device from the connection request sent by the drop receiving module, and the drop module triggers the device management platform to verify the terminal device according to the device information of the terminal device. It should be noted that the delivery module can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
  • the drop module establishes a connection relationship with the drop receiving module, thereby allowing the terminal device to access the server.
  • the drop module may directly allow the terminal device to access the server without verifying the terminal device.
  • the delivery receiving module sends decoder information to the delivery module.
  • the drop receiving module uploads the decoder information of the terminal device to the drop module.
  • the decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
  • the delivery module initializes the system.
  • the delivery module After the delivery module obtains the decoder information from the delivery receiving module, the delivery module confirms the decoding capability of the terminal device according to the decoder information, and creates a thread with the browser, where the thread is used for data interaction between the browser and the delivery module.
  • the drop receiving module sends a media playback request.
  • the delivery receiving module sends a media playback request to the delivery module, where the media playback request includes key instruction information input by the user.
  • the key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices.
  • the key instruction information may be as shown in Table 3-3.
  • the delivery module transmits a media playback request to the browser.
  • the delivery module transmits the media playback request obtained from the delivery receiving module to the browser through a thread.
  • the browser obtains page element images, media information, and streaming media data.
  • the browser After receiving the media playback request, the browser determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the browser loads the media playback page, In order to obtain the page element image, and the media information and streaming media data of the media that the receiving module requests to play.
  • the streaming media data is the media data of the media requested by the media playback request
  • the page element image is the page image of the media playback page corresponding to the media playback request
  • the media information includes the video resolution of the streaming media data, streaming media protocol, audio and video Configuration information such as encoding format and duration.
  • the browser sends page element images and media information to the delivery module.
  • the browser sends page element images and media information to the drop module through threads.
  • the drop module performs pixel processing on the page element image.
  • the delivery module After the delivery module obtains the page element image from the browser, it uses its pixel processing capability to stream the page element image to obtain the streaming data of the media playback page.
  • the streaming data may be information such as a media play operation button, a fast forward icon, a fast backward icon, or a pause icon. It should be noted that the streaming data may be a picture or a video stream.
  • the delivery module sends streaming data and media information to the delivery receiving module.
  • the delivery module sends the streaming data and media information of the media playback page to the delivery receiving module, where the media information is used by the delivery receiving module to create a media player.
  • the delivery and reception module creates a media player and displays a media playback page.
  • the delivery receiving module obtains media information from the delivery module, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data .
  • the delivery receiving module obtains the streaming data of the media playback page from the delivery module, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
  • the media player created by the delivery and receiving module may be embedded in the media playback page.
  • the delivery module receives streaming media data.
  • the delivery module After the delivery module receives the media information, it determines the streaming media protocol from the media information, and receives the streaming media data from the browser according to the streaming media protocol. After the delivery module obtains the streaming media data, the delivery module decapsulates the streaming media data, and The unpacked streaming media data is cached in the first cache area. It should be noted that, during the process of receiving streaming media data, the delivery module may monitor the data volume of the streaming media data buffered in the first buffer area.
  • the drop module separately caches the audio data stream and the video data stream of the streaming media data in the first buffer area.
  • the delivery module sends a receiving instruction to the delivery receiving module.
  • the delivery module sends a receiving instruction to the delivery receiving module, and the receiving instruction is used to instruct the delivery receiving module to receive the streaming media data.
  • the delivery receiving module sends a data acquisition request to the delivery module.
  • the delivery receiving module After receiving the receiving instruction from the delivery module, the delivery receiving module sends a data acquisition request to the delivery module, where the data acquisition request includes the data type of the requested data.
  • the data type may be as shown in Table 3-4.
  • the delivery module sends streaming media data to the delivery receiving module.
  • the placing module obtains the data obtaining request sent by the placing receiving module, and determines the data type of the data requested by the placing receiving module from the data obtaining request.
  • the drop module sends the streaming media data whose coding format is the target coding format to the drop receiving module according to the decoder information of the terminal device, where the target coding format matches the decoder information.
  • the delivery module separately sends the audio data stream and the video data stream of the streaming media data to the delivery receiving module.
  • the placement module may also send a DRM certificate to the placement receiving module, and the DRM certificate is used by the placement receiving module to decrypt the encrypted streaming media data.
  • the drop receiving module caches the streaming media data in the second cache area.
  • the drop receiving module receives streaming media data from the drop module, and caches the streaming media data in the second buffer area of the drop receiving module.
  • the placing and receiving module may monitor the data volume of the streaming media data buffered in the second buffer area. When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the drop receiving module stops requesting the streaming media data.
  • the drop receiving module separately buffers the audio data stream and the video data stream of the streaming media data in the second buffer area.
  • the drop receiving module sends a data reading instruction to the media player.
  • the drop receiving module stops requesting the streaming media data and sends a data reading instruction to the media player.
  • the delivery receiving module may resume receiving streaming media data from the delivery module.
  • the media player caches the streaming media data in the third cache area.
  • the media player obtains the data reading instruction sent by the delivery receiving module.
  • the media player reads the first data from the second buffer area of the delivery receiving module according to the data reading instruction, and buffers the read first data in the third buffer area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area.
  • the media player may provide an interface to the delivery receiving module, and the delivery receiving module may perform the data amount of the first data buffered in the third buffer area through the interface Monitoring, when the data amount of the first data in the third buffer area reaches the third preset threshold, the drop receiving module stops the media player from reading the first data.
  • the media player when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area respectively, and reads them separately according to the PTS pair
  • the audio data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are cached in the third buffer area.
  • the streaming media data when the streaming media data is an ES stream, the streaming media data includes an audio data stream and a video data stream, and the media player reads the audio data stream and the video data stream from the second buffer area, And cached in the third cache area respectively.
  • the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
  • the media player plays the streaming media data.
  • the media player reads the second data from the third buffer area, where the second data is streaming media data.
  • the media player decodes and plays the streaming media data.
  • the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
  • the media player when the streaming media data is an encrypted stream, before the media player decodes and plays the streaming media data, the media player may also obtain a DRM certificate, and use the DRM certificate to decrypt the encrypted streaming media data.
  • the media player when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream from the third buffer area respectively, and performs decoding and playback respectively.
  • the server's delivery and reception module sends the streaming data and streaming media data of the media playback page to the delivery and reception module, respectively, the delivery and reception module of the terminal device displays the media playback page, and the media player of the terminal device performs convection Media data is decoded and played. Therefore, in this embodiment, the media player of the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the drop receiving module of the terminal device, and the server does not need to stream Data and streaming media data are re-encoded, reducing broadcast control delay and improving user experience.
  • FIG. 5 is a schematic diagram of a streaming media data cache provided by an embodiment of the present application. As shown in FIG. ⁇ 502 ⁇ ⁇ Buffer area 503.
  • the first buffer area 501 is a buffer area of the placement module, and is used to buffer the streaming media data after the decapsulation of the placement module.
  • the drop module creates a thread and a first buffer area 501 according to the streaming protocol in the media information, the drop module decapsulates the received streaming media data, and then decapsulates the The streaming media data is cached in the first cache area 501.
  • the delivery module monitors the data volume of the streaming media data cached in the first cache area 501. When the data volume of the streaming media data cached in the first cache area 501 is greater than At a preset threshold, a receiving instruction is sent to the delivery receiving module.
  • the drop module may read the streaming media data from the first buffer area 501 and send the streaming media data to the drop receiving module through the transmission channel.
  • the second buffer area 502 is a buffer area of the placement receiving module, and is used to buffer the streaming media data received from the placement module.
  • the delivery receiving module determines the encoding format of the streaming media data, and creates a second buffer area 502 according to the encoding format.
  • the delivery receiving module may monitor the data volume of the streaming media data buffered in the second buffer area 502, when the streaming media data buffered in the second buffer area 502 When the amount of data is greater than the second preset threshold, the drop receiving module stops requesting streaming media data.
  • the third buffer area 503 is a buffer area of the media player, and is used to buffer the streaming media data read from the second buffer area 502.
  • the streaming media data read from the second buffer area 502 is buffered to the third buffer area 503 to fetch the data.
  • the streaming media data passes through the three-level cache, which effectively guarantees the fluency and integrity of the streaming media data, so that the audio data and video data of the streaming media data are played synchronously.
  • FIG. 6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application.
  • the schematic diagram of the streaming media data cache may be used to cache ES streams.
  • Three levels of cache are performed, including a first cache area 601, a second cache area 602, and a third cache area 603.
  • the first buffer area 601 includes a video buffer area 6011 and an audio buffer area 6012
  • the second buffer area 602 includes a video buffer area 6021 and an audio buffer area 6022
  • the third buffer area 603 includes a video buffer area 6031 and an audio buffer area 6032.
  • the video buffer area 6011 and the audio buffer area 6012 are buffer areas of the delivery module, and the video buffer area 6021 and the audio buffer area 6012 are used to buffer the video data stream and the audio data stream received from the browser, respectively.
  • the delivery module creates a video buffer area 6011 and an audio buffer area 6012 according to the streaming media protocol in the media information.
  • the delivery module requests the audio data stream and the video data stream from the browser respectively.
  • the delivery module will The received video data stream and audio data stream are decapsulated respectively, and then the decapsulated video data stream is buffered in the video buffer area 6011, and the unpacked audio data stream is buffered in the audio buffer area 6012.
  • the delivery module monitors the respective amounts of data corresponding to the video data stream buffered in the video buffer area 6011 and the audio data stream buffered in the audio buffer area 6012.
  • the drop module stops requesting the video data stream.
  • the drop The module stops requesting audio data streams.
  • the video buffer area 6021 and the audio buffer area 6022 are buffer areas of the drop receiving module, and the video buffer area 6021 and the audio buffer area 6022 are used to buffer the video data stream and audio data received from the drop module, respectively.
  • the delivery receiving module determines the encoding format corresponding to the video data stream and audio data, respectively, and creates an audio buffer area and a video buffer area according to the encoding format.
  • the dropping and receiving module may cache according to the type of streaming media data.
  • the audio data stream is cached in the audio buffer area 6022.
  • the video data stream is buffered in the video buffer area 6021.
  • the delivery and receiving module monitors the data amounts corresponding to the video data stream buffered in the video buffer area 6021 and the audio data stream in the audio buffer area 6022 respectively.
  • the video data stream of the stop request is placed, and when the data volume of the audio data stream buffered in the audio buffer area 6022 is greater than the second preset threshold At this time, the drop receiving module stops requesting the audio data stream.
  • the video buffer area 6031 and the audio buffer area 6032 are buffer areas of the media player, wherein the video buffer area 6031 is used to buffer the video data stream read from the video buffer area 6021, and the audio buffer area 6032 is used to buffer the video buffer area 6022 Read audio data stream.
  • the media player reads the video data stream and the audio data stream from the video buffer area 6021 and the audio buffer area 6022 of the second buffer area 602, respectively, and controls The data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are buffered in the video buffer area 6031 and the audio buffer area 6032, respectively.
  • the video data stream and the audio data respectively pass through the three-level cache, which effectively guarantees the smoothness and integrity of the video data stream and the audio data, so that the video data stream and the audio data of the streaming media data are played synchronously.
  • FIG. 7 is a schematic block diagram of a server provided by an embodiment of the present application.
  • the server includes:
  • the obtaining unit 701 is used to obtain a media playback request
  • the processing unit 702 is configured to obtain streaming data of a media playback page, media information and streaming media data of the media requested to be played by the terminal device according to the media playback request, where the media information is configuration information of the streaming media data ;
  • the sending unit 703 is configured to send the streaming data and the media information to the terminal device, the media information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display The media playback page;
  • the sending unit 703 is further configured to send the streaming media data to the terminal device.
  • processing unit 702 is specifically configured to:
  • the streaming media data includes an audio data stream and a video data stream
  • the sending unit 703 is specifically configured to send the audio data stream and the video data stream to the terminal device, respectively.
  • the obtaining unit 701 is further configured to obtain decoder information of the terminal device from a terminal device, where the decoder information includes a decoding format supported by the terminal device;
  • the sending unit 703 is specifically configured to send the streaming media data whose encoding format is a target encoding format to the terminal device, where the target encoding format matches the decoder information.
  • the sending unit 703 is further configured to send a digital rights management DRM certificate to the terminal device, and the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
  • the obtaining unit 701 is further configured to obtain a connection request from the terminal device, where the connection request includes device information of the terminal device;
  • the server further includes a triggering unit 704 and a establishing unit 705, the triggering unit 704 is used to trigger a device management platform to verify the terminal device according to the device information; the establishing unit 705 is used to act as the device management platform When verification of the terminal device is passed, a connection relationship is established with the terminal device.
  • the server further includes a first cache unit 706, and the first cache unit 706 is configured to cache the streaming media data in the first cache area;
  • the sending unit 703 is further configured to send a receiving instruction to the terminal device, and the receiving instruction is used to indicate The terminal device receives the streaming media data.
  • the acquisition unit 701 is further configured to receive a data acquisition request sent by the terminal device, where the data acquisition request includes the data type of the requested data;
  • the sending unit 703 is specifically configured to send the buffered streaming media data to the terminal device according to the data type.
  • the processing unit 702 obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and the sending unit 703 sends the streaming data and streaming media data to the terminal device respectively, and the terminal device
  • the streaming data displays the media playback page, and the terminal device decodes and plays the streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data to reduce the consumption of server resources.
  • the sending unit 703 sends the streaming data and the streaming media data to the terminal device separately, without re-streaming the streaming data and the streaming media data. Encoding reduces broadcast control delay and improves user experience.
  • the server provided by the embodiment of the present application is described above, and the terminal device provided by the embodiment of the present application is described below.
  • FIG. 8 is a schematic block diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 8, the terminal device includes:
  • the sending unit 801 is used to send a media playback request to the server;
  • the receiving unit 802 is configured to receive media information, streaming media data, and streaming data of a media playback page sent by the server, where the media information is configuration information of the streaming media data;
  • a creating unit 803, configured to create a media player based on the media information
  • a display unit 804 configured to display the media playback page based on the streaming data
  • the playing unit 805 is configured to play the streaming media data on the media playing page using the media player.
  • the streaming media data includes an audio data stream and a video data stream
  • the playback unit 805 is specifically configured to play the audio data stream using the media player on the media playback page And the video data stream;
  • the terminal device further includes a synchronization unit 806, which is specifically configured to synchronize the audio data stream and the video data according to the display time PTS of the audio and video frames.
  • the receiving unit 802 is further configured to receive a digital rights management DRM certificate sent by the server;
  • the terminal device further includes a decryption unit 807 for decrypting the streaming media data according to the DRM certificate.
  • the receiving unit 802 is further configured to receive a receiving instruction sent by the server;
  • the sending unit 801 is further configured to send a data acquisition request to the server according to the receiving instruction, the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send the data type corresponding to Streaming data.
  • the terminal device further includes a second cache unit 808, and the second cache unit 808 is configured to cache the streaming media data in the second cache area;
  • the receiving unit 802 is further configured to stop receiving the streaming media data when the data volume of the streaming media data buffered in the second buffer area reaches a second preset threshold.
  • the sending unit 801 is further configured to send decoder information of the terminal device to the server, where the decoder information is used to instruct the server to send the encoding format to the target encoding format
  • the target encoding format matches the decoder information.
  • the sending unit 801 is further configured to send a connection request to the server, where the connection request includes device information of the terminal device, and the device information is used by the server to End device verification.
  • the receiving unit 802 receives the streaming media data and the streaming data of the media playback page sent by the server respectively, and the display unit 804 displays the media playback page according to the streaming data, and the playback unit 805 decodes the streaming media data Play, reduce the broadcast control delay of playing media.
  • FIG. 9 is another schematic block diagram of a server provided by an embodiment of the present application.
  • the server includes a browser 201 and a delivery module 202.
  • the actions performed by the browser 201 are similar to the actions performed by the browser involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again.
  • the actions performed by the delivery module 202 are similar to the actions performed by the delivery module involved in the embodiment corresponding to FIG. 4 described above, and details are not described herein again.
  • FIG. 4 refer to FIG. 4.
  • the functional structures of the browser 201 and the delivery module 202 provided in this embodiment are similar to those described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
  • FIG. 10 is another schematic block diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device includes a drop receiving module 203 and a media player 204.
  • the action performed by the delivery and receiving module 203 is similar to the action performed by the delivery and receiving module involved in the embodiment corresponding to FIG. 4 described above, which will not be repeated here.
  • the actions performed by the media player 204 are similar to the actions performed by the media player involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again.
  • FIG. 4 refer to FIG. 4.
  • the functional structure of the delivery receiving module 203 and the media player 204 provided in this embodiment is similar to that described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
  • FIG. 11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application.
  • the server includes:
  • the transceiver may include a receiver and a transmitter.
  • the memory 1150 may include a read-only memory and / or a random access memory, and provide operation instructions and data to the processor 1110.
  • a portion of the memory 1150 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory and the processor may be independently connected through a bus or an interface, or may be integrated together.
  • the memory 1150 stores the following elements, executable modules or data structures, or their subsets, or their extensions.
  • the corresponding operation is performed by calling the operation instruction stored in the memory 1150 (the operation instruction may be stored in the operating system).
  • the processor 1110 controls the operation of the server, and the processor 1110 may also be called a CPU (Central Processing Unit, central processing unit).
  • the memory 1150 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1110. A portion of the memory 1150 may also include non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the various components of the server are coupled together through a bus system 1120.
  • the bus system 1120 may also include a power bus, a control bus, and a status signal bus. However, for clarity, various buses are marked as the bus system 1120 in the figure.
  • the method disclosed in the above embodiments of the present application may be applied to the processor 1110 or implemented by the processor 1110.
  • the processor 1110 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1110 or an instruction in the form of software.
  • the aforementioned processor 1110 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA off-the-shelf programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied and executed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register.
  • the storage medium is located in the memory 1150.
  • the memory 1150 may be a physically independent unit, or may be integrated with the processor 1110.
  • the processor 1110 reads the information in the memory 1150 and completes the steps of the above method in combination with its hardware.
  • the transceiver 1130 may be used to execute the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4 on the server side related to the receiving and sending operation steps, respectively. Or used to perform the steps of sending and receiving data on the server side in other optional embodiments.
  • the processor 1110 may be used to execute the steps of server-side data processing in the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4, respectively. Or it is used to perform the steps of server-side data processing in other optional embodiments.
  • FIG. 12 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application.
  • FIG. 12 shows a simplified schematic diagram of a hardware structure of a terminal device. It is easy to understand and convenient to illustrate.
  • the terminal device uses a mobile phone as an example.
  • the terminal device includes a processor, a memory, a radio frequency circuit, an antenna, and input and output devices.
  • the processor is mainly used for processing communication protocols and communication data, as well as controlling terminal devices, executing software programs, and processing data of software programs.
  • the memory is mainly used to store software programs and data.
  • the radio frequency circuit is mainly used for the conversion of the baseband signal and the radio frequency signal and the processing of the radio frequency signal.
  • the antenna is mainly used to send and receive radio frequency signals in the form of electromagnetic waves.
  • Input and output devices such as touch screens, display screens, and keyboards, are mainly used to receive data input by users and output data to users. It should be noted that some types of terminal devices may not have input / output devices.
  • the processor When data needs to be sent, the processor performs baseband processing on the data to be sent, and then outputs the baseband signal to the radio frequency circuit.
  • the radio frequency circuit processes the baseband signal after radio frequency processing and sends the radio frequency signal to the outside in the form of electromagnetic waves through the antenna.
  • the radio frequency circuit receives the radio frequency signal through the antenna, converts the radio frequency signal into a baseband signal, and outputs the baseband signal to the processor.
  • the processor converts the baseband signal into data and processes the data.
  • FIG. 12 only one memory and processor are shown in FIG. 12. In actual terminal equipment products, there may be one or more processors and one or more memories.
  • the memory may also be referred to as a storage medium or storage device.
  • the memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.
  • an antenna and a radio frequency circuit with a transceiver function can be regarded as a transceiver unit of a terminal device, and a processor with a processing function can be regarded as a processing unit of the terminal device. among them.
  • the transceiver unit may also be called a transceiver, a transceiver, a transceiver device, or the like.
  • the processing unit may also be called a processor, a processing board, a processing module, a processing device, and the like.
  • the device used to implement the receiving function in the transceiver unit can be regarded as the receiving unit
  • the device used to implement the sending function in the transceiver unit can be regarded as the sending unit
  • the transceiver unit includes a receiving unit and a sending unit.
  • the transceiver unit may sometimes be called a transceiver, transceiver, or transceiver circuit.
  • the receiving unit may sometimes be called a receiver, a receiver, or a receiving circuit.
  • the sending unit may sometimes be called a transmitter, a transmitter, or a transmitting circuit.
  • transceiving unit is used to perform the sending operation and the receiving operation on the terminal device side in the above method embodiment
  • processing unit is used to perform other processing operations on the terminal device in addition to the transceiving operation in the above method embodiment.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a division of logical functions.
  • there may be other divisions for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or software function unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.
  • the technical solution of the present application may be essentially or part of the contribution to the existing technology or all or part of the technical solution may be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

Disclosed is a media processing method, which is used for a server to respectively send streaming data of a media playback page and stream media data of a media which is requested to be played back by a terminal device to the terminal device. The method of embodiments of the present application comprises: the server obtains a media playback request from the terminal device; the server obtains the streaming data of the media playback page and media information and stream media data of the media which is requested to be played back by a terminal device according to the media playback request, the media information being configuration information of the stream media data; the server sends the streaming data and the media information to the terminal device, the media information being used for the terminal device to create a media player, and the streaming data being used for the terminal device to display the media playback page; the server sends the stream media data to the terminal device. Therefore, according to the embodiments of the present application, the server respectively sends the streaming data and the stream media data to the terminal device, and the terminal device displays the media playback page and plays back the stream media data so as to reduce server resource consumption and playback control delay.

Description

媒体处理方法及装置Media processing method and device
本申请要求于2018年10月12日提交中国国家知识产权局、申请号为201811191179.8、发明名称为“媒体处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application requires the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on October 12, 2018 with the application number 201811191179.8 and the invention titled "Media Processing Method and Device", the entire content of which is incorporated by reference .
技术领域Technical field
本申请实施例涉及通信领域，尤其涉及一种媒体处理方法及装置。The embodiments of the present application relate to the communication field, and in particular, to a media processing method and device.
背景技术Background technique
随着技术的发展，云视频成为视频产业的主流之一，其中云视频可基于云化技术实现，基于云化技术的云视频可以通过云端服务器的浏览器加载视频页面，从而得到视频画面，云端服务器再将视频画面进行流化处理并传输至终端设备，由终端设备对视频画面进行播放。With the development of technology, cloud video has become one of the mainstream of the video industry. Cloud video can be implemented based on cloud technology. Cloud video based on cloud technology can load video pages through the browser of the cloud server to obtain video images. Cloud The server then streams the video picture and transmits it to the terminal device, and the terminal device plays the video picture.
现有技术中，云端服务器对于视频画面的每一帧渲染画面都进行视频解码，然后将解码后的渲染画面与页面信息进行合并，并将合并的视频信息与页面信息进行重编码，输出支持终端设备播放的编码格式的视频流，下发至终端设备播放。In the prior art, the cloud server performs video decoding on each frame of the rendered video, and then merges the decoded rendered image with the page information, and re-encodes the merged video information with the page information to output a support terminal. The encoded video stream played by the device is sent to the terminal device for playback.
但是，云端服务器对于视频画面的每一帧渲染画面都进行解码，对云端服务器资源消耗较大，并且云端服务器对解码后的视频信息与页面信息进行合并的视频流进行重编码，增加播控延时，从而影响了用户体验。However, the cloud server decodes every frame of the video frame rendering, which consumes a lot of resources on the cloud server, and the cloud server recodes the video stream that combines the decoded video information and page information to increase the broadcast control delay. Time, thereby affecting the user experience.
发明内容Summary of the invention
本申请实施例提供了一种媒体处理方法及装置，用于服务器分别向终端设备发送媒体播放页面的流化数据和终端设备请求播放的媒体的流媒体数据，并由终端设备显示媒体播放页面以及播放流媒体数据，降低服务器的资源消耗和播控时延。An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
本申请实施例第一方面提供一种媒体处理方法，包括：A first aspect of an embodiment of the present application provides a media processing method, including:
服务器从终端设备获取媒体播放请求；所述服务器根据所述媒体播放请求获取媒体播放页面的流化数据，以及所述终端设备请求播放的媒体的媒体信息和流媒体数据，其中，所述媒体信息为所述流媒体数据的配置信息；所述服务器向所述终端设备发送所述流化数据和所述媒体信息，所述流化数据用于所述终端设备显示所述媒体播放页面所述媒体信息用于所述终端设备创建媒体播放器，所述流化数据用于所述终端设备显示所述媒体播放页面；所述服务器向所述终端设备发送所述流媒体数据，以使所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据。由第一方面可见，服务器将媒体播放页面的流化数据和终端设备请求播放的媒体的流媒体数据分别向终端设备发送，并由终端设备显示媒体播放页面以及播放流媒体数据，使得服务器无需对流化数据和流媒体数据进行解码以及合并，降低服务器的资源消耗和播控时延。The server obtains the media playback request from the terminal device; the server obtains the streaming data of the media playback page according to the media playback request, and the media information and streaming media data of the media requested by the terminal device to play, where Is the configuration information of the streaming media data; the server sends the streaming data and the media information to the terminal device, and the streaming data is used by the terminal device to display the media on the media playback page The information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display the media playback page; the server sends the streaming media data to the terminal device to enable the terminal The device uses the media player to play the streaming media data on the media playing page. As can be seen from the first aspect, the server sends the streaming data of the media playback page and the streaming media data of the media requested by the terminal device to the terminal device, and the terminal device displays the media playback page and plays the streaming media data, so that the server does not need to Decode and merge the streaming data and streaming media data to reduce server resource consumption and broadcast control delay.
基于本申请实施例第一方面，本申请实施例第一方面的第一种实现方式中，所述服务器根据所述媒体播放请求获取媒体播放页面的流化数据、所述终端设备请求播放的媒体的媒体信息和流媒体数据包括：所述服务器加载所述媒体播放请求对应的所述媒体播放页面， 对所述媒体播放页面进行流化得到所述媒体播放页面的流化数据；所述服务器从所述媒体播放页面获取所述媒体信息和所述流媒体数据。由第一方面的第一种实现方式可见，通过加载所述媒体播放请求对应的所述媒体播放页面，从而可以得到所述流化数据以及所述媒体信息和所述流媒体数据，提供一种获取所述流化数据以及所述媒体信息和所述流媒体数据的具体方法，提升了方案的可实现性。Based on the first aspect of the embodiment of the present application, in a first implementation manner of the first aspect of the embodiment of the present application, the server obtains streaming data of a media playback page according to the media playback request, and media requested by the terminal device to play The media information and streaming media data include: the server loads the media playback page corresponding to the media playback request, and streams the media playback page to obtain streaming data of the media playback page; The media playback page obtains the media information and the streaming media data. It can be seen from the first implementation manner of the first aspect that by loading the media playback page corresponding to the media playback request, the streaming data, the media information, and the streaming media data can be obtained to provide a The specific method of acquiring the streaming data, the media information, and the streaming media data improves the feasibility of the solution.
基于本申请实施例第一方面以及第一方面的第一种实现方式，本申请实施例第一方面的第二种实现方式中，所述流媒体数据包括音频数据流和视频数据流，所述服务器向所述终端设备发送所述流媒体数据包括：所述服务器分别向所述终端设备发送所述音频数据流和所述视频数据流。由第一方面的第二种实现方式可见，将流媒体数据的音频数据流和视频数据流分别向终端设备发送，提高了音频数据流和视频数据流的可靠性。Based on the first aspect of the embodiments of the present application and the first implementation manner of the first aspect, in a second implementation of the first aspect of the embodiments of the present application, the streaming media data includes an audio data stream and a video data stream. The server sending the streaming media data to the terminal device includes the server sending the audio data stream and the video data stream to the terminal device, respectively. It can be seen from the second implementation manner of the first aspect that the audio data stream and the video data stream of the streaming media data are separately sent to the terminal device, which improves the reliability of the audio data stream and the video data stream.
基于本申请实施例第一方面以及第一方面的第一种实现方式至第一方面的第二种实现方式中的任一项，本申请实施例第一方面的第三种实现方式中，所述服务器向所述终端设备发送所述流媒体数据之前，所述方法还包括：所述服务器从终端设备获取所述终端设备的解码器信息，所述解码器信息包括所述终端设备支持的解码格式；所述服务器向所述终端设备发送所述流媒体数据包括：所述服务器向所述终端设备发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。由第一方面的第三种实现方式可见，服务器向终端设备发送编码格式为目标编码格式的流媒体数据，使得终端设备可以使用相匹配的解码器对流媒体数据进行解码播放。Based on any one of the first aspect of the embodiments of the present application and the first implementation manner of the first aspect to the second implementation manner of the first aspect, in a third implementation manner of the first aspect of the embodiments of the present application, all Before the server sends the streaming media data to the terminal device, the method further includes: the server acquiring decoder information of the terminal device from the terminal device, the decoder information including decoding supported by the terminal device Format; the server sending the streaming media data to the terminal device includes the server sending the streaming media data to the terminal device in an encoding format of a target encoding format, the target encoding format and the decoder Information matching. It can be seen from the third implementation manner of the first aspect that the server sends the streaming media data in the encoding format of the target encoding format to the terminal device, so that the terminal device can decode and play the streaming media data using a matching decoder.
基于本申请实施例第一方面以及第一方面的第一种实现方式至第一方面的第三种实现方式中的任一项，本申请实施例第一方面的第四种实现方式中,所述流媒体数据为加密流，所述方法还包括：所述服务器向所述终端设备发送数字版权管理DRM证书，所述DRM证书用于所述终端设备对加密的所述流媒体数据进行解密。Based on any one of the first aspect and the first implementation manner of the first aspect to the third implementation manner of the first aspect, in the fourth implementation manner of the first aspect of the embodiment of the present application, all The streaming media data is an encrypted stream, and the method further includes the server sending a digital rights management DRM certificate to the terminal device, where the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
基于本申请实施例第一方面以及第一方面的第一种实现方式至第一方面的第四种实现方式中的任一项，本申请实施例第一方面的第五种实现方式中,所述服务器从终端设备获取所述终端设备的解码器信息之前，所述方法还包括：所述服务器从所述终端设备获取连接请求，所述连接请求包括所述终端设备的设备信息；所述服务器根据所述设备信息触发设备管理平台对所述终端设备进行验证；当所述设备管理平台对所述终端设备验证通过时，所述服务器与所述终端设备建立连接关系。由第一方面的第五种实现方式可见，通过触发设备管理平台对终端设备进行验证，只有验证通过的终端设备才会保持与服务器的连接关系，提升了安全性。Based on any one of the first aspect of the embodiments of the present application and the first implementation manner of the first aspect to the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect of the embodiments of the present application, all Before the server obtains the decoder information of the terminal device from the terminal device, the method further includes: the server obtains a connection request from the terminal device, the connection request includes device information of the terminal device; the server Triggering a device management platform to verify the terminal device according to the device information; when the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device. It can be seen from the fifth implementation of the first aspect that by triggering the device management platform to verify the terminal device, only the terminal device that passes the verification will maintain the connection relationship with the server, which improves security.
基于本申请实施例第一方面以及第一方面的第一种实现方式至第一方面的第五种实现方式中的任一项，本申请实施例第一方面的第六种实现方式中,所述服务器向所述终端设备发送所述流媒体数据之前，所述方法还包括：所述服务器缓存所述流媒体数据于第一缓存区；当缓存于所述第一缓存区的所述流媒体数据的数据量达到第一预置阈值时，所述服务器向所述终端设备发送接收指令，所述接收指令用于指示所述终端设备接收所述流媒体数据。由第一方面的第六种实现方式可见，将流媒体数据缓存于第一缓存区，提升了流媒体数据的完整性和流畅性。Based on any one of the first aspect of the embodiments of the present application and the first implementation manner of the first aspect to the fifth implementation manner of the first aspect, in the sixth implementation manner of the first aspect of the embodiments of the present application, all Before the server sends the streaming media data to the terminal device, the method further includes: the server caches the streaming media data in a first cache area; when the streaming media is cached in the first cache area When the data amount of the data reaches the first preset threshold, the server sends a reception instruction to the terminal device, where the reception instruction is used to instruct the terminal device to receive the streaming media data. It can be seen from the sixth implementation manner of the first aspect that the streaming media data is cached in the first buffer area, which improves the integrity and fluency of the streaming media data.
基于本申请实施例第一方面以及第一方面的第一种实现方式至第一方面的第六种实现 方式中的任一项，本申请实施例第一方面的第七种实现方式中,所述服务器向所述终端设备发送所述流媒体数据包括：所述服务器接收所述终端设备发送的数据获取请求，所述数据获取请求包括请求数据的数据类型；所述服务器根据所述数据类型向所述终端设备发送缓存的所述流媒体数据。由第一方面的第七种实现方式可见，服务器根据数据请求的数据类型向终端设备发送流媒体数据，提升了发送流媒体数据的准确性。Based on any one of the first aspect and the first implementation manner of the first aspect to the sixth implementation manner of the first aspect, in the seventh implementation manner of the first aspect of the embodiment of the present application, all The server sending the streaming media data to the terminal device includes: the server receiving a data acquisition request sent by the terminal device, the data acquisition request includes a data type of the requested data; The terminal device sends the buffered streaming media data. It can be seen from the seventh implementation of the first aspect that the server sends streaming media data to the terminal device according to the data type of the data request, which improves the accuracy of sending streaming media data.
本申请实施例第二方面提供一种媒体处理方法，包括：A second aspect of an embodiment of the present application provides a media processing method, including:
终端设备获取到用户输入的媒体播放请求时，所述终端设备向服务器发送媒体播放请求；所述终端设备接收所述服务器发送的媒体信息和媒体播放页面的流化数据，所述媒体信息为所述流媒体数据的配置信息；所述终端设备基于所述媒体信息创建媒体播放器；所述终端设备基于所述流化数据显示所述媒体播放页面；所述终端设备接收服务器发送的流媒体数据，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据。由第二方面可见，终端设备分别对媒体播放页面的流化数据进行解码显示，以及对流媒体数据进行解码播放，降低了播控时延。When the terminal device obtains the media playback request input by the user, the terminal device sends a media playback request to the server; the terminal device receives the media information sent by the server and the streaming data of the media playback page, and the media information is Configuration information of the streaming media data; the terminal device creates a media player based on the media information; the terminal device displays the media playback page based on the streaming data; the terminal device receives streaming media data sent by the server , The terminal device uses the media player to play the streaming media data on the media playing page. It can be seen from the second aspect that the terminal device separately decodes and displays the streaming data of the media playback page and decodes and plays the streaming media data, thereby reducing the broadcast control delay.
基于本申请实施例第二方面，本申请实施例第二方面的第一种实现方式中，所述流媒体数据包括音频数据流和视频数据流，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据包括：所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述音频数据流和所述视频数据流；所述终端设备根据音视频帧的显示时间PTS对所述音频数据流和所述视频数据进行同步。由第二方面的第一种实现方式可见，终端设备根据PTS对所述音频数据流和所述视频数据进行同步，使得所述音频数据流和所述视频数据的可以在媒体播放器中同步播放。Based on the second aspect of the embodiment of the present application, in a first implementation manner of the second aspect of the embodiment of the present application, the streaming media data includes an audio data stream and a video data stream, and the terminal device uses the media playback page The playing of the streaming media data by the media player includes: the terminal device using the media player to play the audio data stream and the video data stream on the media playing page; the terminal device according to the audio and video The frame display time PTS synchronizes the audio data stream and the video data. It can be seen from the first implementation of the second aspect that the terminal device synchronizes the audio data stream and the video data according to the PTS, so that the audio data stream and the video data can be played synchronously in the media player .
基于本申请实施例第二方面以及第二方面的第一种实现方式，本申请实施例第二方面的第二种实现方式中，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据之前，所述方法还包括：所述终端设备接收所述服务器发送的数字版权管理DRM证书；所述终端设备根据所述DRM证书对所述流媒体数据进行解密。由第二方面的第二种实现方式可见，终端设备通过使用DRM证书对所述流媒体数据进行解密，提高了流媒体数据的安全性。Based on the second aspect of the embodiments of the present application and the first implementation of the second aspect, in a second implementation of the second aspect of the embodiments of the present application, the terminal device uses the media playback on the media playback page Before the player plays the streaming media data, the method further includes: the terminal device receives a digital rights management DRM certificate sent by the server; and the terminal device decrypts the streaming media data according to the DRM certificate. It can be seen from the second implementation manner of the second aspect that the terminal device decrypts the streaming media data by using a DRM certificate, thereby improving the security of the streaming media data.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第二种实现方式中的任一项，本申请实施例第二方面的第三种实现方式中，终端设备接收所述服务器发送的所述流媒体数据之前，所述方法还包括：所述终端设备接收所述服务器发送的接收指令；所述终端设备根据所述接收指令向所述服务器发送数据获取请求，所述数据获取请求包括请求数据的数据类型，所述数据类型用于指示所述服务器发送所述数据类型对应的流媒体数据。Based on any one of the second aspect of the embodiments of the present application and the first implementation manner of the second aspect to the second implementation manner of the second aspect, in a third implementation manner of the second aspect of the embodiment of the present application, the terminal Before the device receives the streaming media data sent by the server, the method further includes: the terminal device receives a receiving instruction sent by the server; the terminal device sends a data acquisition request to the server according to the receiving instruction The data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send streaming media data corresponding to the data type.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第三种实现方式中的任一项，本申请实施例第二方面的第四种实现方式中，终端设备接收所述服务器发送的所述流媒体数据之后，所述方法还包括：所述终端设备缓存所述流媒体数据于第二缓存区；当缓存于所述第二缓存区所述流媒体数据的数据量达到第二预置阈值时，所述终端设备停止接收所述流媒体数据。由第二方面的第四种实现方式可见，终端设备将流媒体缓存于第二缓存区中，提升了流媒体数据的完整性和流畅性。Based on any one of the second aspect of the embodiments of the present application and any one of the first implementation manner of the second aspect to the third implementation manner of the second aspect, in a fourth implementation manner of the second aspect of the embodiment of the present application, the terminal After the device receives the streaming media data sent by the server, the method further includes: the terminal device caches the streaming media data in a second cache area; when the streaming media data is cached in the second cache area When the amount of data reaches the second preset threshold, the terminal device stops receiving the streaming media data. It can be seen from the fourth implementation of the second aspect that the terminal device caches the streaming media in the second buffer area, which improves the integrity and fluency of the streaming media data.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第四种实现方式中的任一项，本申请实施例第二方面的第五种实现方式中，所述终端设备向服务器发送媒体播放请求之前，所述方法还包括：所述终端设备向所述服务器发送所述终端设备的解码器信息，所述解码器信息用于指示所述服务器发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。由第二方面的第五种实现方式可见，终端设备向服务器发送解码器信息，使得请求得到的数据的编码格式与终端设备的解码器信息相匹配。Based on any one of the second aspect of the embodiments of the present application and the first implementation manner of the second aspect to the fourth implementation manner of the second aspect, in the fifth implementation manner of the second aspect of the embodiments of the present application, all Before the terminal device sends a media playback request to the server, the method further includes: the terminal device sends decoder information of the terminal device to the server, and the decoder information is used to instruct the server to send the encoding format as The streaming media data in a target encoding format, the target encoding format matches the decoder information. It can be seen from the fifth implementation manner of the second aspect that the terminal device sends decoder information to the server so that the encoding format of the requested data matches the decoder information of the terminal device.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第五种实现方式中的任一项，本申请实施例第二方面的第六种实现方式中，所述终端设备向所述服务器发送所述终端设备的解码器信息之前，所述方法还包括：所述终端设备向所述服务器发送连接请求，所述连接请求包括所述终端设备的设备信息，所述设备信息用于所述服务器对所述终端设备进行验证。由第二方面的第六种实现方式可见，终端设备通过向服务器发送设备信息，使得服务器可以根据设备信息触发对终端设备进行验证，提升了安全性。Based on any one of the second aspect of the embodiments of the present application and the first implementation manner of the second aspect to the fifth implementation manner of the second aspect, in the sixth implementation manner of the second aspect of the embodiments of the present application, all Before the terminal device sends the decoder information of the terminal device to the server, the method further includes: the terminal device sends a connection request to the server, and the connection request includes device information of the terminal device. The device information is used by the server to verify the terminal device. It can be seen from the sixth implementation manner of the second aspect that the terminal device sends device information to the server, so that the server can trigger verification of the terminal device based on the device information, thereby improving security.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第六种实现方式中的任一项，本申请实施例第二方面的第七种实现方式中，所述流媒体数据包括所述音频数据流和所述视频数据流，所述终端设备缓存所述音视频数据于第二缓存区包括：所述终端设备分别缓存所述音频数据流和所述视频数据流于第二缓存区。Based on any one of the second aspect and the first implementation manner of the second aspect to the sixth implementation manner of the second aspect, in the seventh implementation manner of the second aspect of the embodiment of the present application, all The streaming media data includes the audio data stream and the video data stream, and the terminal device buffering the audio and video data in the second buffer area includes: the terminal device buffering the audio data stream and the video data, respectively Flow in the second buffer area.
基于本申请实施例第二方面以及第二方面的第一种实现方式至第二方面的第七种实现方式中的任一项，本申请实施例第二方面的第八种实现方式中，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据包括：所述终端设备从所述第二缓存区读取到第一数据，并将所述第一数据缓存于所述媒体播放器的第三缓存区；所述终端设备从所述第三缓存区中读取到第二数据，并在所述媒体播放页面上使用所述媒体播放器播放所述第二数据。由第二方面的第八种实现方式可见，终端设备将第一数据缓存至媒体播放器的第三缓存区中，使得媒体播放器可以直接从第三缓存区中读取第二数据进行播放，提升了流媒体数据的完整性和流畅性。Based on any one of the second aspect and the first implementation manner of the second aspect to the seventh implementation manner of the second aspect, in the eighth implementation manner of the second aspect of the embodiment of the present application, all The terminal device using the media player to play the streaming media data on the media playback page includes: the terminal device reading the first data from the second cache area and caching the first data In the third cache area of the media player; the terminal device reads the second data from the third cache area and uses the media player to play the second data on the media playback page data. It can be seen from the eighth implementation of the second aspect that the terminal device caches the first data in the third cache area of the media player, so that the media player can directly read the second data from the third cache area for playback, Improve the integrity and fluency of streaming media data.
本申请实施例第三方面提供一种服务器，所述服务器包括：存储器、收发器和至少一个处理器，所述存储器中存储有指令；所述存储器、所述收发器和所述至少一个处理器通过线路连接；A third aspect of an embodiment of the present application provides a server. The server includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one processor Connected by line;
所述至少一个处理器调用所述指令，执行第一方面在所述服务器侧进行的消息处理或控制操作。The at least one processor calls the instruction to perform the message processing or control operation performed on the server side in the first aspect.
本申请实施例第四方面提供一种终端设备，所述终端设备包括：存储器、收发器和至少一个处理器，所述存储器中存储有指令；所述存储器、所述收发器和所述至少一个处理器通过线路连接；A fourth aspect of an embodiment of the present application provides a terminal device. The terminal device includes: a memory, a transceiver, and at least one processor, and the memory stores instructions; the memory, the transceiver, and the at least one The processor is connected by line;
所述至少一个处理器调用所述指令，执行第二方面在所述终端设备侧进行的消息处理或控制操作。The at least one processor calls the instruction to perform the message processing or control operation performed on the terminal device side in the second aspect.
本申请实施例第五方面提供一种计算机可读存储介质，其特征在于，包括指令，当所述指令在计算机上运行时，使得计算机执行第一方面或第一方面任一可能实现方式的方法。A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the first aspect or any possible implementation manner of the first aspect .
本申请实施例第六方面提供一种计算机可读存储介质，其特征在于，包括指令，当所述指令在计算机上运行时，使得计算机执行第二方面或第二方面任一可能实现方式的方法。A sixth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions, which when executed on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect .
本申请实施例第七方面提供一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，使得所述计算机执行第一方面或第一方面任一可能实现方式的方法。A seventh aspect of the embodiments of the present application provides a computer program product containing instructions, characterized in that, when it is run on a computer, the computer is caused to execute the method of the first aspect or any possible implementation manner of the first aspect.
本申请实施例第八方面提供一种包含指令的计算机程序产品，其特征在于，当其在计算机上运行时，使得所述计算机执行第二方面或第二方面任一可能实现方式的方法。An eighth aspect of an embodiment of the present application provides a computer program product containing instructions, which, when it runs on a computer, causes the computer to execute the method of the second aspect or any possible implementation manner of the second aspect.
从以上技术方案可以看出，本申请实施例具有以下优点：It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:
本申请实施例中，服务器根据终端设备发送的媒体播放请求获取媒体播放页面的流化数据和流媒体数据，并将流化数据和流媒体数据分别向终端设备发送，由终端设备根据流化数据显示媒体播放页面，以及由终端设备对流媒体数据进行解码播放。因此，本实施例中由终端设备对流媒体数据进行解码播放，降低了服务器资源的消耗，同时将流化数据和流媒体数据分别向终端设备发送，无须对流化数据和流媒体数据进行重编码，降低了播控时延，提高了用户体验。In the embodiment of the present application, the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device according to the streaming data Display the media playback page, and decode and play the streaming media data by the terminal device. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
附图说明BRIEF DESCRIPTION
图1为本申请实施例提供的一个应用场景示意图；FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application;
图2为本申请实施例提供的一个***框架图；2 is a system framework diagram provided by an embodiment of this application;
图3为本申请实施例提供的媒体处理方法的一个实施例示意图；3 is a schematic diagram of an embodiment of a media processing method provided by an embodiment of the present application;
图4为本申请实施例提供的媒体处理方法的另一个实施例示意图；4 is a schematic diagram of another embodiment of a media processing method provided by an embodiment of this application;
图5为本申请实施例提供的一个流媒体数据缓存的示意图；5 is a schematic diagram of a streaming media data cache provided by an embodiment of this application;
图6为本申请实施例提供的另一个流媒体数据缓存的示意图；6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application;
图7为本申请实施例提供的服务器的一个示意性框图；7 is a schematic block diagram of a server provided by an embodiment of this application;
图8为本申请实施例提供的终端设备的一个示意性框图；8 is a schematic block diagram of a terminal device provided by an embodiment of this application;
图9为本申请实施例提供的服务器的另一个示意性框图；9 is another schematic block diagram of a server provided by an embodiment of this application;
图10为本申请实施例提供的终端设备的另一个示意性框图；10 is another schematic block diagram of a terminal device provided by an embodiment of this application;
图11为本申请实施例提供的服务器的一个硬件结构示意图；11 is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application;
图12为本申请实施例提供的终端设备的一个硬件结构示意图。12 is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application.
具体实施方式detailed description
下面结合附图，对本申请的实施例进行描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。本领域普通技术人员可知，随着新技术的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。The following describes the embodiments of the present application with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. It can be known by those of ordinary skill in the art that with the emergence of new technologies, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
本申请实施例的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请实施例的实施例 如能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the embodiments of the present application and the above drawings are used to distinguish similar objects, and It is not necessary to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, processes, methods, systems, products or devices that contain a series of steps or units need not be limited to those clearly listed Those steps or units, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or equipment.
本申请实施例提供了一种媒体处理方法及装置，用于服务器分别向终端设备发送媒体播放页面的流化数据和终端设备请求播放的媒体的流媒体数据，并由终端设备显示媒体播放页面以及播放流媒体数据，降低服务器的资源消耗和播控时延。An embodiment of the present application provides a media processing method and apparatus for a server to respectively send streaming data of a media playback page and streaming media data of a media requested by a terminal device to a terminal device, and the terminal device displays the media playback page and Play streaming media data to reduce server resource consumption and broadcast control delay.
请参考图1，图1为本申请实施例提供的一个应用场景示意图，如图1所示，该应用场景示意图可以包括服务器101以及终端设备102。Please refer to FIG. 1, which is a schematic diagram of an application scenario provided by an embodiment of the present application. As shown in FIG. 1, the schematic diagram of the application scenario may include a server 101 and a terminal device 102.
其中，服务器101可以作为媒体投放方，终端设备102可以作为媒体被投放方。终端设备102向服务器101发送媒体播放请求，服务器101根据媒体播放请求分别向终端设备102发送媒体播放页面的流化数据、以及终端设备102请求播放的媒体的媒体信息和流媒体数据，然后由终端设备102显示媒体播放页面，以及根据媒体信息创建媒体播放器，并使媒体播放器对流媒体数据进行播放。Among them, the server 101 may serve as a media delivery party, and the terminal device 102 may serve as a media delivery party. The terminal device 102 sends a media playback request to the server 101. The server 101 sends the streaming data of the media playback page and the media information and streaming media data of the media requested by the terminal device 102 to the terminal device 102 according to the media playback request, and then the terminal The device 102 displays a media playback page, and creates a media player according to the media information, and causes the media player to play streaming media data.
本申请实施例提供的终端设备102可以包括各种具有无线通信功能的手持设备、车载设备、可穿戴设备、计算设备或连接到无线调制解调器的其它处理设备。所述终端设备102可以是机顶盒、智能电视、移动站(mobile station，MS)、用户单元(subscriber unit)、蜂窝电话(cellular phone)、智能电话(smart phone)、无线数据卡、个人数字助理(personal digital assistant，PDA)电脑、平板型电脑、无线调制解调器(modem)、手持设备(handset)、膝上型电脑(laptop computer)、机器类型通信(machine type communication,MTC)终端等。The terminal device 102 provided in the embodiments of the present application may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem. The terminal device 102 may be a set-top box, a smart TV, a mobile station (MS), a subscriber unit (subscriber unit), a cellular phone (cellular phone), a smart phone (smart phone), a wireless data card, and a personal digital assistant ( Personal digital assistant (PDA) computer, tablet computer, wireless modem (modem), handheld device (handset), laptop computer (laptop computer), machine type communication (machine type communication (MTC) terminal, etc.
本申请实施例提供的服务器101可以是云端服务器，该云端服务器具有高度分布式、高度虚拟化的特点。其中，云端服务器可以将媒体播放请求获取得到的媒体播放页面的流化数据和流媒体数据通过远程投放的方式发送至终端设备102。The server 101 provided in the embodiment of the present application may be a cloud server, and the cloud server has the characteristics of being highly distributed and highly virtualized. The cloud server may send the streaming data and streaming media data of the media playback page obtained by the media playback request to the terminal device 102 in a remote delivery mode.
需要说明的是，本申请实施例中，媒体播放请求可以是Web视频播请求，其中，Web视频播请求请求播放的视频为Web视频。也就是说云端服务器可以将Web视频播请求对应视频播放页面的流化数据，以及Web视频的流媒体数据通过远程投放的方式发送至终端设备102。It should be noted that, in the embodiment of the present application, the media playback request may be a Web video broadcast request, where the video requested by the Web video broadcast request is Web video. That is to say, the cloud server can send the streaming data corresponding to the video playback page of the Web video broadcast request and the streaming media data of the Web video to the terminal device 102 by remote drop.
请参考图2，图2为本申请实施例提供的一个***框架图，该***框架图包括服务器以及终端设备。Please refer to FIG. 2, which is a system framework diagram provided by an embodiment of the present application. The system framework diagram includes a server and a terminal device.
本申请实施例中，服务器可作为媒体投放方，服务器可以包括浏览器201和投放模块202，终端设备可作为媒体被投放方，终端设备包括投放接收模块203和媒体播放器204。In the embodiment of the present application, the server may serve as a media delivery party, the server may include a browser 201 and a delivery module 202, the terminal device may serve as a media recipient, and the terminal device includes a delivery receiving module 203 and a media player 204.
浏览器201可以用于加载媒体播请求对应的媒体播放页面和对媒体播放页面中媒体信息进行侦测。浏览器201加载媒体播请求对应的媒体播放页面，可得到页面元素图像，以及投放接收模块请求播放的媒体的媒体信息以及流媒体数据，其中，媒体信息为流媒体数据的配置信息。同时浏览器201向投放模块202输出页面元素图像、媒体信息以及流媒体数据。The browser 201 may be used to load a media playback page corresponding to the media playback request and detect media information in the media playback page. The browser 201 loads the media playback page corresponding to the media broadcast request, and can obtain page element images, and media information and streaming media data of the media requested to be played by the receiving module, where the media information is configuration information of the streaming media data. At the same time, the browser 201 outputs page element images, media information, and streaming media data to the delivery module 202.
投放模块202可以用于接收并处理投放接收模块203的交互指令，以及向投放接收模块203下发媒体信息、媒体播放页面的流化数据和流媒体数据。投放模块202具备像素处 理能力和流媒体处理能力，像素处理能力是将从浏览器201获取的页面元素图像进行流化，以得到媒体播放页面的流化数据，流媒体处理能力是使用对流媒体协议从浏览器201获取封装的流媒体数据，并解除流媒体协议封装，将解封装的流媒体数据缓存于缓存区。The delivery module 202 may be used to receive and process the interactive instruction of the delivery receiving module 203, and deliver media information, streaming data and media data of the media playback page to the delivery receiving module 203. The delivery module 202 has pixel processing capability and streaming media processing capability. The pixel processing capability is to stream the page element image obtained from the browser 201 to obtain streaming data of the media playback page, and the streaming media processing capability is to use the streaming media protocol Obtain the encapsulated streaming media data from the browser 201, and decapsulate the streaming media protocol, and cache the decapsulated streaming media data in the cache area.
投放接收模块203可以用于与投放模块202的交互，以及接收媒体信息、媒体播放页面的流化数据和流媒体数据。The delivery receiving module 203 may be used for interaction with the delivery module 202, as well as receiving media information, streaming data and streaming media data of media playback pages.
媒体播放器204可以使用本地的多媒体处理能力，对流媒体数据进行解码，并将解码后的音视频信息同步等流程，还原在服务器进行的播放过程，从而实现媒体的播放。The media player 204 can use the local multimedia processing capabilities to decode the streaming media data, and synchronize the decoded audio and video information and other processes to restore the playback process performed on the server, thereby achieving media playback.
上面对本实施例中提供的应用场景示意图和***框架图进行了描述，下面对本申请实例提供的媒体处理方法进行描述。The application scenario schematic diagram and system framework diagram provided in this embodiment are described above, and the media processing method provided by the example of the present application is described below.
请参考图3，本申请实施例提供的一种媒体处理方法，可以包括以下步骤：Referring to FIG. 3, a media processing method provided by an embodiment of the present application may include the following steps:
301、终端设备向服务器发送连接请求。301. The terminal device sends a connection request to the server.
在播放媒体之前，终端设备向服务器发起连接请求，该连接请求携带终端设备的设备信息。其中，该设备信息可以包括终端设备的物理地址(MAC Addr)、终端设备的出厂标识符(UID)和登录终端设备的用户的用户名等信息。具体地，该设备信息可以如表3-1所示。Before playing the media, the terminal device initiates a connection request to the server, and the connection request carries the device information of the terminal device. Wherein, the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device. Specifically, the device information may be as shown in Table 3-1.
表3-1.终端设备的设备信息Table 3-1. Device information of terminal devices
Figure PCTCN2019106327-appb-000001
Figure PCTCN2019106327-appb-000001
302、服务器触发设备管理平台对终端设备进行验证。302. The server triggers the device management platform to verify the terminal device.
服务器从终端设备发送的连接请求中获取终端设备的设备信息，服务器根据该设备信息触发设备管理平台对终端设备进行验证。需要说明的是服务器还可以根据设备信息触发运营商对终端设备进行验证，此处不做限定。The server acquires the device information of the terminal device from the connection request sent by the terminal device, and the server triggers the device management platform to verify the terminal device according to the device information. It should be noted that the server can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
当设备管理平台对终端设备验证通过时，服务器与终端设备建立连接关系，允许终端设备接入服务器。When the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device, allowing the terminal device to access the server.
本实施例在实际应用中，服务器也可以无需对终端设备进行验证，直接允许终端设备接入服务器。In practical applications of this embodiment, the server may directly allow the terminal device to access the server without verifying the terminal device.
303、终端设备向服务器发送解码器信息。303. The terminal device sends decoder information to the server.
终端设备接入服务器之后，终端设备向服务器上传终端设备的解码器信息，其中解码器信息可以包括终端设备支持的音频编码格式、视频编码格式和字幕格式等信息。具体地，该解码器信息可以如表3-2所示。After the terminal device is connected to the server, the terminal device uploads the decoder information of the terminal device to the server, where the decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
表3-2.解码器信息Table 3-2. Decoder information
Figure PCTCN2019106327-appb-000002
Figure PCTCN2019106327-appb-000002
Figure PCTCN2019106327-appb-000003
Figure PCTCN2019106327-appb-000003
304、服务器初始化***。304. The server initializes the system.
服务器从终端设备获取解码器信息之后，服务器根据该解码器信息确认终端设备的解码能力并初始化***。After the server obtains the decoder information from the terminal device, the server confirms the decoding capability of the terminal device and initializes the system according to the decoder information.
305、终端设备发送媒体播放请求。305. The terminal device sends a media playback request.
终端设备向服务器发送媒体播放请求，该媒体播放请求包括用户输入的按键指令信息。其中该按键指令信息可以是用户通过遥控器或者键盘鼠标等设备输入，该按键指令信息可以如表3-3所示。The terminal device sends a media playback request to the server, where the media playback request includes key instruction information input by the user. The key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices. The key instruction information may be as shown in Table 3-3.
表3-3.按键指令信息Table 3-3. Key command information
属性Attributes 取值样例Sample value 取值意义Value meaning
键值Key-value KEY_OKKEY_OK 按键值Key value
按键事件Key event KEY_DOWNKEY_DOWN 按下/抬起Press / lift
306、服务器获取流化数据、媒体信息和流媒体数据。306. The server obtains streaming data, media information, and streaming media data.
服务器从媒体播放请求中确定按键指令信息，并对该按键指令信息进行逻辑处理，从而可以确定该媒体播放请求对应的媒体播放页面，服务器加载该媒体播放页面，并对媒体播放页面进行流化以得到该媒体播放页面的流化数据，以及从媒体播放页面获取终端设备请求播放的媒体的媒体信息和流媒体数据。其中，流媒体数据为媒体播放请求请求播放的媒体的媒体数据，流化数据为媒体播放请求对应媒体播放页面的页面数据，该流化数据可以为媒体的播放操作按钮、快进图标、快退图标或暂停图标等信息。需要说明的是，该流化数据可以是图片或者视频流。媒体信息包括流媒体数据的视频分辨率、流媒体协议、音视频编码格式和时长等配置信息。The server determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the server loads the media playback page and streams the media playback page to Obtain the streaming data of the media playback page, and obtain the media information and streaming media data of the media requested to be played by the terminal device from the media playback page. Among them, the streaming media data is the media data of the media requested by the media playback request, and the streaming data is the page data of the media playback page corresponding to the media playback request. The streaming data can be a media playback operation button, a fast forward icon, and a fast backward Information such as icons or pause icons. It should be noted that the streaming data may be a picture or a video stream. The media information includes configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data.
307、服务器将流媒体数据缓存于第一缓存区。307. The server caches the streaming media data in the first cache area.
服务器将从媒体播放页面获取的流媒体数据缓存于第一缓存区，并对缓存于第一缓存区的流媒体数据的数据量进行监测。The server caches the streaming media data acquired from the media playback page in the first cache area, and monitors the data volume of the streaming media data cached in the first cache area.
需要说明的是，当流媒体数据为基本码流(elementary stream，ES)流时，服务器分别将流媒体数据的音频数据流和视频数据流缓存于第一缓存区。It should be noted that when the streaming media data is an elementary stream (ES) stream, the server caches the audio data stream and the video data stream of the streaming media data in the first buffer area, respectively.
308、服务器发送流化数据和媒体信息。308. The server sends streaming data and media information.
服务器向终端设备发送媒体播放页面的流化数据和媒体信息，其中，媒体信息用于终端设备创建媒体播放器。The server sends the streaming data and media information of the media playback page to the terminal device, where the media information is used by the terminal device to create a media player.
309、终端设备创建媒体播放器和显示媒体播放页面。309. The terminal device creates a media player and displays a media playback page.
终端设备从服务器获取媒体信息，并从媒体信息中确定流媒体数据的视频分辨率、流媒体协议、音视频编码格式和时长等配置信息，然后根据流媒体数据的配置信息创建媒体播放器，该媒体播放器播放流媒体数据对应的音视频信息。The terminal device obtains the media information from the server, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data. The media player plays the audio and video information corresponding to the streaming media data.
终端设备从服务器获取媒体播放页面的流化数据，并根据该流化数据显示媒体播放页面，其中媒体播放页面可用于对媒体的进行操作。例如，媒体播放页面包括媒体的快进图标时，终端设备在媒体播放页面上显示快进图标，用户可以通过该快进图标对媒体进行快进操作。The terminal device obtains the streaming data of the media playback page from the server, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
需要说明的是，本实施例中终端设备创建的媒体播放器可嵌入至媒体播放页面中。It should be noted that, in this embodiment, the media player created by the terminal device may be embedded in the media playback page.
310、服务器向终端设备发送接收指令。310. The server sends a reception instruction to the terminal device.
当缓存于第一缓存区的流媒体数据的数据量达到第一预置阈值时，服务器向终端设备发送接收指令，接收指令用于指示终端设备接收流媒体数据。When the data volume of the streaming media data buffered in the first buffer area reaches the first preset threshold, the server sends a receiving instruction to the terminal device, and the receiving instruction is used to instruct the terminal device to receive the streaming media data.
311、终端设备向服务器发送数据获取请求。311. The terminal device sends a data acquisition request to the server.
终端设备从服务器获取到接收指令之后，向服务器发送数据获取请求，其中，该数据获取请求包括请求数据的数据类型。具体地，该数据类型具体可以如表3-4所示。After acquiring the receiving instruction from the server, the terminal device sends a data acquisition request to the server, where the data acquisition request includes the data type of the requested data. Specifically, the data type may be as shown in Table 3-4.
表3-4.请求数据的数据类型Table 3-4. Data types of requested data
属性Attributes 取值样例Sample value 取值意义Value meaning
数据类型type of data videovideo 请求数据的数据类型The data type of the requested data
312、服务器向终端设备发送流媒体数据。312. The server sends streaming media data to the terminal device.
服务器获取终端设备发送的数据获取请求，从数据获取请求中确定终端设备请求数据的数据类型。The server acquires the data acquisition request sent by the terminal device, and determines the data type of the data requested by the terminal device from the data acquisition request.
服务器根据所述终端设备的解码器信息，向终端设备发送编码格式为目标编码格式的流媒体数据，其中目标编码格式与解码器信息匹配。The server sends streaming media data whose encoding format is the target encoding format to the terminal device according to the decoder information of the terminal device, where the target encoding format matches the decoder information.
需要说明的是，本实施例中，当流媒体数据为ES流时，服务器分别向终端设备发送流媒体数据的音频数据流和视频数据流。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the server separately sends the audio data stream and the video data stream of the streaming media data to the terminal device.
可选地，本实施例中当流媒体数据为加密流时，服务器还可以向终端设备发送数字版权管理(digital rights management，DRM)证书,DRM证书用于终端设备对加密的流媒体数据进行解密。Optionally, in this embodiment, when the streaming media data is an encrypted stream, the server may also send a digital rights management (digital rights management (DRM) certificate to the terminal device. The DRM certificate is used by the terminal device to decrypt the encrypted streaming media data .
313、终端设备将流媒体数据缓存于第二缓存区。313. The terminal device caches the streaming media data in the second cache area.
终端设备从服务器接收流媒体数据，并将流媒体数据缓存于终端设备的第二缓存区中。其中终端设备在接收流媒体数据的过程中，终端设备可以对缓存于第二缓存区的流媒体数据的数据量进行监测，当缓存于第二缓存区的流媒体数据的数据量达到第二预置阈值时，终端设备停止请求流媒体数据。The terminal device receives streaming media data from the server, and caches the streaming media data in the second buffer area of the terminal device. During the process of receiving the streaming media data, the terminal device can monitor the data volume of the streaming media data cached in the second cache area, and when the data volume of the streaming media data cached in the second cache area reaches the second pre- When the threshold is set, the terminal device stops requesting streaming media data.
可以理解的是，当缓存于第二缓存区的流媒体数据的数据量小于第二预置阈值时，终端设备可以恢复接收流媒体数据。It can be understood that, when the amount of streaming media data buffered in the second buffer area is less than the second preset threshold, the terminal device may resume receiving streaming media data.
需要说明的是，本实施例中，当流媒体数据为ES流时，终端设备分别缓存流媒体数据的音频数据流和视频数据流于第二缓存区。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the terminal device separately caches the audio data stream and the video data stream of the streaming media data in the second buffer area.
314、终端设备将流媒体数据缓存于第三缓存区。314. The terminal device caches the streaming media data in the third buffer area.
当缓存于第二缓存区的流媒体数据的数据量达到第二预置阈值时，终端设备从第二缓存区中读取第一数据，并将读取的第一数据缓存于媒体播放器的第三缓存区。需要说明的是，第一数据包含于缓存在第二缓存区的流媒体数据，第一数据可以是缓存在第二缓存区的流媒体数据的一部分或者全部。在读取第一数据的过程中，终端设备对缓存于第三缓存区的第一数据的数据量进行监测，当第三缓存区的第一数据的数据量达到第三预置阈值时，终端设备停止读取第一数据。When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the terminal device reads the first data from the second buffer area, and buffers the read first data in the media player The third cache area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area. In the process of reading the first data, the terminal device monitors the data amount of the first data buffered in the third buffer area, and when the data amount of the first data in the third buffer area reaches the third preset threshold, the terminal The device stops reading the first data.
需要说明的是，本实施例中，当流媒体数据为ES流时，终端设备分别从第二缓存区中 读取缓存流媒体数据的音频数据流和视频数据流之后，终端设备可以根据音视频帧的显示时间(presentation time stamp，PTS)对音频数据流和视频数据流进行同步，将同步后的音频数据流和视频数据流缓存于第三缓存区。It should be noted that, in this embodiment, when the streaming media data is an ES stream, after the terminal device reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area, the terminal device The presentation time (PTS) of the frame synchronizes the audio data stream and the video data stream, and buffers the synchronized audio data stream and video data stream in the third buffer area.
需要说明的是，本实施例中，当流媒体数据为ES流时，第一数据包括音频数据流和视频数据流，终端设备从第二缓存区分别读取音频数据流和视频数据流，并分别缓存于第三缓存区。It should be noted that in this embodiment, when the streaming media data is an ES stream, the first data includes an audio data stream and a video data stream, and the terminal device reads the audio data stream and the video data stream from the second buffer area, Cache in the third cache area respectively.
可以理解的是，本实施中流媒体数据通过第一缓存区、第二缓存区和第三缓存区的三级缓存，保证了流媒体数据的流畅性和完整性。It can be understood that, in this implementation, the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
315、终端设备使用媒体播放器播放流媒体数据。315. The terminal device uses a media player to play streaming media data.
终端设备从第三缓存区中读取第二数据，其中，第二数据为流媒体数据。终端设备使用媒体播放器对流媒体数据进行解码播放。需要说明的是，第二数据包含于缓存在第三缓存区的第一数据，第二数据可以是缓存于第三缓存区的第一数据的一部分或者全部，也就是说第二数据为流媒体数据的一部分或者全部。The terminal device reads the second data from the third buffer area, where the second data is streaming media data. The terminal device uses a media player to decode and play the streaming media data. It should be noted that the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
可选地，本实施例中当流媒体数据为加密流时，在终端设备对第二数据进行解码播放之前，终端设备还可以获取服务器发送DRM证书,并使用DRM证书对加密的流媒体数据进行解密。Optionally, in this embodiment, when the streaming media data is an encrypted stream, before the terminal device decodes and plays the second data, the terminal device may also obtain a DRM certificate sent by the server and use the DRM certificate to perform encryption on the streaming media data Decrypt.
本实施例中，服务器根据终端设备发送的媒体播放请求获取媒体播放页面的流化数据和流媒体数据，并将流化数据和流媒体数据分别向终端设备发送，由终端设备根据流化数据显示媒体播放页面，以及由终端设备对流媒体数据进行解码播放。因此，本实施例中由终端设备对流媒体数据进行解码播放，降低了服务器资源的消耗，同时将流化数据和流媒体数据分别向终端设备发送，无须对流化数据和流媒体数据进行重编码，降低了播控时延，提高了用户体验。In this embodiment, the server obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and sends the streaming data and streaming media data to the terminal device separately, and the terminal device displays according to the streaming data Media playback page, and the terminal device decodes and plays streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the terminal device without re-encoding the streaming data and the streaming media data. Reduced broadcast control delay and improved user experience.
上面对本实施例中提供的一种媒体处理方法进行了描述，下面对本申请实例提供的另一种媒体处理方法进行描述。The media processing method provided in this embodiment has been described above, and another media processing method provided by an example of the present application is described below.
请参考图4，本申请实施例提供的另一种媒体处理方法，可以包括以下步骤：Referring to FIG. 4, another media processing method provided by an embodiment of the present application may include the following steps:
401、投放接收模块向投放模块发送连接请求。401. The delivery receiving module sends a connection request to the delivery module.
在播放媒体之前，投放接收模块向投放模块发起连接请求，该连接请求携带终端设备的设备信息。其中，该设备信息可以包括终端设备的物理地址(MAC Addr)、终端设备的出厂标识符(UID)和登录终端设备的用户的用户名等信息。具体地，该设备信息可以如表3-1所示。Before playing the media, the drop receiving module initiates a connection request to the drop module, the connection request carrying device information of the terminal device. Wherein, the device information may include information such as the physical address (MAC Addr) of the terminal device, the factory identifier (UID) of the terminal device, and the user name of the user who logs in to the terminal device. Specifically, the device information may be as shown in Table 3-1.
402、投放模块触发设备管理平台对终端设备进行验证。402. The drop module triggers the device management platform to verify the terminal device.
投放模块从投放接收模块发送的连接请求中获取终端设备的设备信息，投放模块根据终端设备的设备信息触发设备管理平台对终端设备进行验证。需要说明的是投放模块还可以根据设备信息触发运营商对终端设备进行验证，此处不做限定。The drop module obtains the device information of the terminal device from the connection request sent by the drop receiving module, and the drop module triggers the device management platform to verify the terminal device according to the device information of the terminal device. It should be noted that the delivery module can also trigger the operator to verify the terminal device based on the device information, which is not limited here.
当设备管理平台对终端设备验证通过时，投放模块与投放接收模块建立连接关系，从而使得允许终端设备接入服务器。When the device management platform passes the verification of the terminal device, the drop module establishes a connection relationship with the drop receiving module, thereby allowing the terminal device to access the server.
本实施例在实际应用中，投放模块也可以无需对终端设备进行验证，直接允许终端设备接入服务器。In practical applications of this embodiment, the drop module may directly allow the terminal device to access the server without verifying the terminal device.
403、投放接收模块向投放模块发送解码器信息。403. The delivery receiving module sends decoder information to the delivery module.
终端设备接入服务器之后，投放接收模块向投放模块上传终端设备的解码器信息，该解码器信息可以包括终端设备支持的音频编码格式、视频编码格式和字幕格式等信息。具体地，该解码器信息可以如表3-2所示。After the terminal device is connected to the server, the drop receiving module uploads the decoder information of the terminal device to the drop module. The decoder information may include information such as audio encoding format, video encoding format, and subtitle format supported by the terminal device. Specifically, the decoder information may be as shown in Table 3-2.
404、投放模块初始化***。404. The delivery module initializes the system.
投放模块从投放接收模块获取解码器信息之后，投放模块根据该解码器信息确认终端设备的解码能力，并与浏览器创建线程，其中，该线程用于浏览器与投放模块之间的数据交互。After the delivery module obtains the decoder information from the delivery receiving module, the delivery module confirms the decoding capability of the terminal device according to the decoder information, and creates a thread with the browser, where the thread is used for data interaction between the browser and the delivery module.
405、投放接收模块发送媒体播放请求。405. The drop receiving module sends a media playback request.
投放接收模块向投放模块发送媒体播放请求，该媒体播放请求包括用户输入的按键指令信息。其中该按键指令信息可以是用户通过遥控器或者键盘鼠标等设备输入，该按键指令信息可以如表3-3所示。The delivery receiving module sends a media playback request to the delivery module, where the media playback request includes key instruction information input by the user. The key instruction information may be input by a user through a remote controller, a keyboard, a mouse, and other devices. The key instruction information may be as shown in Table 3-3.
406、投放模块向浏览器传输媒体播放请求。406. The delivery module transmits a media playback request to the browser.
投放模块通过线程向浏览器传输从投放接收模块获取的媒体播放请求。The delivery module transmits the media playback request obtained from the delivery receiving module to the browser through a thread.
407、浏览器获取页面元素图像、媒体信息和流媒体数据。407. The browser obtains page element images, media information, and streaming media data.
浏览器接收到媒体播放请求之后，从媒体播放请求中确定按键指令信息，并对该按键指令信息进行逻辑处理，从而可以确定该媒体播放请求对应的媒体播放页面，浏览器加载该媒体播放页面，以得到页面元素图像、以及投放接收模块请求播放的媒体的媒体信息和流媒体数据。其中，流媒体数据为媒体播放请求请求播放的媒体的媒体数据，页面元素图像为媒体播放请求对应的媒体播放页面的页面图像，媒体信息包括流媒体数据的视频分辨率、流媒体协议、音视频编码格式和时长等配置信息。After receiving the media playback request, the browser determines the key instruction information from the media playback request and performs logic processing on the key instruction information, so that the media playback page corresponding to the media playback request can be determined, and the browser loads the media playback page, In order to obtain the page element image, and the media information and streaming media data of the media that the receiving module requests to play. The streaming media data is the media data of the media requested by the media playback request, the page element image is the page image of the media playback page corresponding to the media playback request, and the media information includes the video resolution of the streaming media data, streaming media protocol, audio and video Configuration information such as encoding format and duration.
408、浏览器向投放模块发送页面元素图像和媒体信息。408. The browser sends page element images and media information to the delivery module.
浏览器通过线程向投放模发送页面元素图像和媒体信息。The browser sends page element images and media information to the drop module through threads.
409、投放模块对页面元素图像进行像素处理。409. The drop module performs pixel processing on the page element image.
投放模块从浏览器获取到页面元素图像之后，使用其像素处理能力对页面元素图像进行流化，以得到媒体播放页面的流化数据。其中，该流化数据可以是媒体的播放操作按钮、快进图标、快退图标或暂停图标等信息。需要说明的是，流化数据可以是图片或者视频流。After the delivery module obtains the page element image from the browser, it uses its pixel processing capability to stream the page element image to obtain the streaming data of the media playback page. Wherein, the streaming data may be information such as a media play operation button, a fast forward icon, a fast backward icon, or a pause icon. It should be noted that the streaming data may be a picture or a video stream.
410、投放模块向投放接收模块发送流化数据和媒体信息。410. The delivery module sends streaming data and media information to the delivery receiving module.
投放模块向投放接收模块发送媒体播放页面的流化数据和媒体信息，其中，媒体信息用于投放接收模块创建媒体播放器。The delivery module sends the streaming data and media information of the media playback page to the delivery receiving module, where the media information is used by the delivery receiving module to create a media player.
411、投放接收模块创建媒体播放器和显示媒体播放页面。411. The delivery and reception module creates a media player and displays a media playback page.
投放接收模块从投放模块获取媒体信息，并从媒体信息中确定流媒体数据的视频分辨率、流媒体协议、音视频编码格式和时长等配置信息，然后根据流媒体数据的配置信息创建媒体播放器。The delivery receiving module obtains media information from the delivery module, and determines the configuration information such as the video resolution, streaming media protocol, audio and video encoding format and duration of the streaming media data from the media information, and then creates a media player according to the configuration information of the streaming media data .
投放接收模块从投放模块获取媒体播放页面的流化数据，并根据流化数据显示媒体播放页面，其中媒体播放页面可用于对媒体的进行操作。例如，媒体播放页面包括媒体的快进图标时，终端设备在媒体播放页面上显示快进图标，用户可以通过该快进图标对媒体进 行快进操作。The delivery receiving module obtains the streaming data of the media playback page from the delivery module, and displays the media playback page according to the streaming data, where the media playback page can be used to operate the media. For example, when the media playback page includes a fast-forward icon for media, the terminal device displays a fast-forward icon on the media playback page, and the user can perform fast-forward operations on the media through the fast-forward icon.
需要说明的是，本实施例中投放接收模块创建的媒体播放器可嵌入至媒体播放页面中。It should be noted that, in this embodiment, the media player created by the delivery and receiving module may be embedded in the media playback page.
412、投放模块接收流媒体数据。412. The delivery module receives streaming media data.
投放模块接收媒体信息之后，从媒体信息中确定流媒体协议，并根据该流媒体协议从浏览器接收流媒体数据，投放模块获取到流媒体数据后，投放模块对流媒体数据进行解封装，并将解封装后的流媒体数据缓存于第一缓存区中。需要说明的是，在接收流媒体数据的过程中，投放模块可以对缓存于第一缓存区的流媒体数据的数据量进行监测。After the delivery module receives the media information, it determines the streaming media protocol from the media information, and receives the streaming media data from the browser according to the streaming media protocol. After the delivery module obtains the streaming media data, the delivery module decapsulates the streaming media data, and The unpacked streaming media data is cached in the first cache area. It should be noted that, during the process of receiving streaming media data, the delivery module may monitor the data volume of the streaming media data buffered in the first buffer area.
本实施例中，当流媒体数据为ES流时，投放模块分别缓存流媒体数据的音频数据流和视频数据流存于第一缓存区。In this embodiment, when the streaming media data is an ES stream, the drop module separately caches the audio data stream and the video data stream of the streaming media data in the first buffer area.
413、投放模块向投放接收模块发送接收指令。413. The delivery module sends a receiving instruction to the delivery receiving module.
当缓存于第一缓存区的流媒体数据的数据量达到第一预置阈值时，投放模块向投放接收模块发送接收指令，接收指令用于指示投放接收模块接收流媒体数据。When the data volume of the streaming media data buffered in the first buffer area reaches the first preset threshold, the delivery module sends a receiving instruction to the delivery receiving module, and the receiving instruction is used to instruct the delivery receiving module to receive the streaming media data.
414、投放接收模块向投放模块发送数据获取请求。414. The delivery receiving module sends a data acquisition request to the delivery module.
投放接收模块从投放模块获取到接收指令之后，向投放模块发送数据获取请求，其中，该数据获取请求包括请求数据的数据类型。具体地，该数据类型具体可以如表3-4所示。After receiving the receiving instruction from the delivery module, the delivery receiving module sends a data acquisition request to the delivery module, where the data acquisition request includes the data type of the requested data. Specifically, the data type may be as shown in Table 3-4.
415、投放模块向投放接收模块发送流媒体数据。415. The delivery module sends streaming media data to the delivery receiving module.
投放模块获取投放接收模块发送的数据获取请求，从数据获取请求中确定投放接收模块请求数据的数据类型。The placing module obtains the data obtaining request sent by the placing receiving module, and determines the data type of the data requested by the placing receiving module from the data obtaining request.
投放模块根据所述终端设备的解码器信息，向投放接收模块发送编码格式为目标编码格式的流媒体数据，其中目标编码格式与解码器信息匹配。The drop module sends the streaming media data whose coding format is the target coding format to the drop receiving module according to the decoder information of the terminal device, where the target coding format matches the decoder information.
需要说明的是，本实施例中，当流媒体数据为ES流时，投放模块分别向投放接收模块发送流媒体数据的音频数据流和视频数据流。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the delivery module separately sends the audio data stream and the video data stream of the streaming media data to the delivery receiving module.
可选地，本实施例中当流媒体数据为加密流时，投放模块还可以向投放接收模块发送DRM证书,DRM证书用于投放接收模块对加密的流媒体数据进行解密。Optionally, in this embodiment, when the streaming media data is an encrypted stream, the placement module may also send a DRM certificate to the placement receiving module, and the DRM certificate is used by the placement receiving module to decrypt the encrypted streaming media data.
416、投放接收模块将流媒体数据缓存于第二缓存区。416. The drop receiving module caches the streaming media data in the second cache area.
投放接收模块从投放模块接收流媒体数据，并将流媒体数据缓存于投放接收模块的第二缓存区中。其中投放接收模块在接收流媒体数据的过程中，投放接收模块可以对缓存于第二缓存区的流媒体数据的数据量进行监测。当缓存于第二缓存区的流媒体数据的数据量达到第二预置阈值时，投放接收模块停止请求流媒体数据。The drop receiving module receives streaming media data from the drop module, and caches the streaming media data in the second buffer area of the drop receiving module. During the process of receiving the streaming media data, the placing and receiving module may monitor the data volume of the streaming media data buffered in the second buffer area. When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the drop receiving module stops requesting the streaming media data.
需要说明的是，本实施例中，当流媒体数据为ES流时，投放接收模块分别缓存流媒体数据的音频数据流和视频数据流于第二缓存区。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the drop receiving module separately buffers the audio data stream and the video data stream of the streaming media data in the second buffer area.
417、投放接收模块向媒体播放器发送数据读取指令。417. The drop receiving module sends a data reading instruction to the media player.
当缓存于第二缓存区的流媒体数据的数据量达到第二预置阈值时，投放接收模块停止请求流媒体数据，并向媒体播放器发送数据读取指令。When the data volume of the streaming media data buffered in the second buffer area reaches the second preset threshold, the drop receiving module stops requesting the streaming media data and sends a data reading instruction to the media player.
可以理解的是，当缓存于第二缓存区的流媒体数据的数据量小于第二预置阈值时，投放接收模块可以恢复从投放模块接收流媒体数据。It can be understood that, when the data volume of the streaming media data buffered in the second buffer area is less than the second preset threshold, the delivery receiving module may resume receiving streaming media data from the delivery module.
418、媒体播放器将流媒体数据缓存于第三缓存区。418. The media player caches the streaming media data in the third cache area.
媒体播放器获取投放接收模块发送的数据读取指令。The media player obtains the data reading instruction sent by the delivery receiving module.
媒体播放器根据数据读取指令从投放接收模块的第二缓存区中读取第一数据，并将读取的第一数据缓存于第三缓存区。需要说明的是，第一数据包含于缓存在第二缓存区的流媒体数据，第一数据可以是缓存在第二缓存区的流媒体数据的一部分或者全部。The media player reads the first data from the second buffer area of the delivery receiving module according to the data reading instruction, and buffers the read first data in the third buffer area. It should be noted that the first data is included in the streaming media data cached in the second cache area, and the first data may be part or all of the streaming media data cached in the second cache area.
本实施例中媒体播放器在读取第一数据的过程中，媒体播放器可以提供接口给投放接收模块，投放接收模块可以通过该接口对缓存于第三缓存区的第一数据的数据量进行监测，当第三缓存区的第一数据的数据量达到第三预置阈值时，投放接收模块停止媒体播放器读取第一数据。In the process of reading the first data in the media player in this embodiment, the media player may provide an interface to the delivery receiving module, and the delivery receiving module may perform the data amount of the first data buffered in the third buffer area through the interface Monitoring, when the data amount of the first data in the third buffer area reaches the third preset threshold, the drop receiving module stops the media player from reading the first data.
需要说明的是，本实施例中当流媒体数据为ES流时，媒体播放器分别从第二缓存区中读取缓存流媒体数据的音频数据流和视频数据流，并根据PTS对分别读取的音频数据流和视频数据流进行同步，将同步后的音频数据流和视频数据流缓存于第三缓存区。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream of the buffered streaming media data from the second buffer area respectively, and reads them separately according to the PTS pair The audio data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are cached in the third buffer area.
需要说明的是，本实施例中，当流媒体数据为ES流时，流媒体数据包括音频数据流和视频数据流，媒体播放器从第二缓存区分别读取音频数据流和视频数据流，并分别缓存于第三缓存区。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the streaming media data includes an audio data stream and a video data stream, and the media player reads the audio data stream and the video data stream from the second buffer area, And cached in the third cache area respectively.
可以理解的是，本实施中流媒体数据通过第一缓存区、第二缓存区和第三缓存区的三级缓存，保证了流媒体数据的流畅性和完整性。It can be understood that, in this implementation, the streaming media data passes through the three-level cache of the first cache area, the second cache area, and the third cache area, which ensures the smoothness and integrity of the streaming media data.
419、媒体播放器对流媒体数据进行播放。419. The media player plays the streaming media data.
媒体播放器从第三缓存区中读取第二数据，其中，第二数据为流媒体数据。媒体播放器对流媒体数据进行解码播放。需要说明的是，第二数据包含于缓存在第三缓存区的第一数据，第二数据可以是缓存在第三缓存区的第一数据的一部分或者全部，也就是说第二数据为流媒体数据的一部分或者全部。The media player reads the second data from the third buffer area, where the second data is streaming media data. The media player decodes and plays the streaming media data. It should be noted that the second data is included in the first data cached in the third cache area, and the second data may be part or all of the first data cached in the third cache area, that is, the second data is streaming media Part or all of the data.
可选地，本实施例中当流媒体数据为加密流时，在媒体播放器对流媒体数据解码播放之前，媒体播放器还可以获取DRM证书,并使用DRM证书对加密的流媒体数据进行解密。Optionally, in this embodiment, when the streaming media data is an encrypted stream, before the media player decodes and plays the streaming media data, the media player may also obtain a DRM certificate, and use the DRM certificate to decrypt the encrypted streaming media data.
需要说明的是，本实施例中，当流媒体数据为ES流时，媒体播放器从第三缓存区分别读取音频数据流和视频数据流，并分别进行解码播放。It should be noted that, in this embodiment, when the streaming media data is an ES stream, the media player reads the audio data stream and the video data stream from the third buffer area respectively, and performs decoding and playback respectively.
本实施例中，服务器的投放接收模块将媒体播放页面的流化数据和流媒体数据分别向投放接收模块发送，由终端设备的投放接收模块显示媒体播放页面，以及由终端设备的媒体播放器对流媒体数据进行解码播放。因此，本实施例中由终端设备的媒体播放器对流媒体数据进行解码播放，降低了服务器资源的消耗，同时将流化数据和流媒体数据分别向终端设备的投放接收模块发送，服务器无须对流化数据和流媒体数据进行重编码，降低了播控时延，提高了用户体验。In this embodiment, the server's delivery and reception module sends the streaming data and streaming media data of the media playback page to the delivery and reception module, respectively, the delivery and reception module of the terminal device displays the media playback page, and the media player of the terminal device performs convection Media data is decoded and played. Therefore, in this embodiment, the media player of the terminal device decodes and plays the streaming media data, which reduces the consumption of server resources. At the same time, the streaming data and the streaming media data are separately sent to the drop receiving module of the terminal device, and the server does not need to stream Data and streaming media data are re-encoded, reducing broadcast control delay and improving user experience.
上面从媒体处理方法对本申请实施例进行了描述，下面从流媒体数据缓存的角度对本申请实施例进行描述。The embodiments of the present application are described above from the media processing method, and the embodiments of the present application are described below from the perspective of streaming media data caching.
请参考图5，图5为本申请实施例提供的一个流媒体数据缓存的示意图，如图5所示，本申请实施例对流媒体数据进行三级缓存，包括第一缓存区501、第二缓存区502和第三缓存区503。Please refer to FIG. 5, which is a schematic diagram of a streaming media data cache provided by an embodiment of the present application. As shown in FIG.区 502 和 第一 Buffer area 503.
第一缓存区501为投放模块的缓存区，用于缓存投放模块解封装之后的流媒体数据。 在终端设备与服务之间建立连接之后，投放模块根据媒体信息中的流媒体协议创建线程和第一缓存区501，投放模块将接收到的流媒体数据进行解封装，然后再将解封后的流媒体数据缓存至第一缓存区501。投放模块在接收到流媒体数据过程中，投放模块对缓存于第一缓存区501中的流媒体数据的数据量进行监控，当缓存于第一缓存区501中的流媒体数据的数据量大于第一预设阈值时，向投放接收模块发送接收指令。当投放接收模块请求数据时，投放模块可以从第一缓存区501中读取流媒体数据，并通过传输通道向投放接收模块发送流媒体数据。The first buffer area 501 is a buffer area of the placement module, and is used to buffer the streaming media data after the decapsulation of the placement module. After the connection between the terminal device and the service is established, the drop module creates a thread and a first buffer area 501 according to the streaming protocol in the media information, the drop module decapsulates the received streaming media data, and then decapsulates the The streaming media data is cached in the first cache area 501. During the process of receiving the streaming media data, the delivery module monitors the data volume of the streaming media data cached in the first cache area 501. When the data volume of the streaming media data cached in the first cache area 501 is greater than At a preset threshold, a receiving instruction is sent to the delivery receiving module. When the drop receiving module requests data, the drop module may read the streaming media data from the first buffer area 501 and send the streaming media data to the drop receiving module through the transmission channel.
第二缓存区502为投放接收模块的缓存区，用于缓存从投放模块接收的流媒体数据。在投放模块发送流媒体数据之后，投放接收模块确定流媒体数据的编码格式，并根据编码格式创建第二缓存区502。投放接收模块接收投放发送的流媒体数据的过程中，投放接收模块可以对缓存于第二缓存区502中的流媒体数据的数据量进行监控，当缓存于第二缓存区502中的流媒体数据的数据量大于第二预设阈值时，投放接收模块停止请求流媒体数据。The second buffer area 502 is a buffer area of the placement receiving module, and is used to buffer the streaming media data received from the placement module. After the delivery module sends the streaming media data, the delivery receiving module determines the encoding format of the streaming media data, and creates a second buffer area 502 according to the encoding format. During the process of receiving the streaming media data sent by the delivery receiving module, the delivery receiving module may monitor the data volume of the streaming media data buffered in the second buffer area 502, when the streaming media data buffered in the second buffer area 502 When the amount of data is greater than the second preset threshold, the drop receiving module stops requesting streaming media data.
第三缓存区503为媒体播放器的缓存区，用于缓存从第二缓存区502中读取的流媒体数据。当获取到投放接收模块发送的数据读取指令时，从第二缓存区502中读取的流媒体数据并缓存至第三缓存区503取数据。The third buffer area 503 is a buffer area of the media player, and is used to buffer the streaming media data read from the second buffer area 502. When the data reading instruction sent by the delivery receiving module is obtained, the streaming media data read from the second buffer area 502 is buffered to the third buffer area 503 to fetch the data.
本实施例中流媒体数据通过三级缓存，有效保证了流媒体数据的流畅性和完整性，从而使得流媒体数据的音频数据和视频数据的同步播放。In this embodiment, the streaming media data passes through the three-level cache, which effectively guarantees the fluency and integrity of the streaming media data, so that the audio data and video data of the streaming media data are played synchronously.
请参考图6，图6为本申请实施例提供的另一个流媒体数据缓存的示意图，该流媒体数据缓存的示意图可以用于缓存ES流，如图6所示，本申请实施例对流媒体数据进行三级缓存，包括第一缓存区601、第二缓存区602和第三缓存区603。其中，第一缓存区601包括视频缓存区6011和音频缓存区6012，第二缓存区602包括视频缓存区6021和音频缓存区6022，第三缓存区603包括视频缓存区6031和音频缓存区6032。Please refer to FIG. 6. FIG. 6 is a schematic diagram of another streaming media data cache provided by an embodiment of the present application. The schematic diagram of the streaming media data cache may be used to cache ES streams. As shown in FIG. Three levels of cache are performed, including a first cache area 601, a second cache area 602, and a third cache area 603. The first buffer area 601 includes a video buffer area 6011 and an audio buffer area 6012, the second buffer area 602 includes a video buffer area 6021 and an audio buffer area 6022, and the third buffer area 603 includes a video buffer area 6031 and an audio buffer area 6032.
视频缓存区6011和音频缓存区6012为投放模块的缓存区，视频缓存区6021和音频缓存区6012分别用于缓存从浏览器接收的视频数据流和音频数据流。在终端设备与服务之间建立连接之后，投放模块根据媒体信息中的流媒体协议创建视频缓存区6011和音频缓存区6012，投放模块分别向浏览器请求音频数据流以及视频数据流，投放模块将接收到的视频数据流和音频数据流分别进行解封装，然后将解封后的视频数据流缓存于视频缓存区6011，以及将解封后的音频数据流缓存于音频缓存区6012。投放模块在接收到视频数据流和音频数据流过程中，投放模块对缓存于视频缓存区6011的视频数据流和缓存于音频缓存区6012的音频数据流分别对应的数据量进行监控，当缓存于视频缓存区6011的视频数据流的数据量大于第一预设阈值时，投放模块停止请求视频数据流，当缓存于音频缓存区6012的音频数据流的数据量大于第一预设阈值时，投放模块停止请求音频数据流。The video buffer area 6011 and the audio buffer area 6012 are buffer areas of the delivery module, and the video buffer area 6021 and the audio buffer area 6012 are used to buffer the video data stream and the audio data stream received from the browser, respectively. After the connection between the terminal device and the service is established, the delivery module creates a video buffer area 6011 and an audio buffer area 6012 according to the streaming media protocol in the media information. The delivery module requests the audio data stream and the video data stream from the browser respectively. The delivery module will The received video data stream and audio data stream are decapsulated respectively, and then the decapsulated video data stream is buffered in the video buffer area 6011, and the unpacked audio data stream is buffered in the audio buffer area 6012. When the delivery module receives the video data stream and the audio data stream, the delivery module monitors the respective amounts of data corresponding to the video data stream buffered in the video buffer area 6011 and the audio data stream buffered in the audio buffer area 6012. When the data volume of the video data stream in the video buffer area 6011 is greater than the first preset threshold, the drop module stops requesting the video data stream. When the data volume of the audio data stream buffered in the audio buffer area 6012 is greater than the first preset threshold, the drop The module stops requesting audio data streams.
视频缓存区6021和音频缓存区6022为投放接收模块的缓存区，视频缓存区6021和音频缓存区6022分别用于缓存从投放模块接收的视频数据流和音频数据。在投放模块发送视频数据流和音频数据之后，投放接收模块确定视频数据流和音频数据分别对应的编码格式，并根据编码格式创建音频缓存区和视频缓存区。投放接收模块接收投放模块发送的流媒体数据的过程中，投放接收模块可以根据流媒体数据的类型进行缓存，当接收到的是音频数据流时，将音频数据流缓存于音频缓存区6022，当接收到的是视频数据流时，将视频数据 流缓存于视频缓存区6021。同时，投放接收模块对缓存于视频缓存区6021的视频数据流和音频缓存区6022音频数据流分别对应的数据量分别进行监控。当缓存于视频缓存区6021的视频数据流的数据量大于第二预设阈值时，投放接收停止请求视频数据流，当缓存于音频缓存区6022的音频数据流的数据量大于第二预设阈值时，投放接收模块停止请求音频数据流。The video buffer area 6021 and the audio buffer area 6022 are buffer areas of the drop receiving module, and the video buffer area 6021 and the audio buffer area 6022 are used to buffer the video data stream and audio data received from the drop module, respectively. After the delivery module sends the video data stream and audio data, the delivery receiving module determines the encoding format corresponding to the video data stream and audio data, respectively, and creates an audio buffer area and a video buffer area according to the encoding format. During the process of receiving the streaming media data sent by the dropping module, the dropping and receiving module may cache according to the type of streaming media data. When the audio data stream is received, the audio data stream is cached in the audio buffer area 6022. When the video data stream is received, the video data stream is buffered in the video buffer area 6021. At the same time, the delivery and receiving module monitors the data amounts corresponding to the video data stream buffered in the video buffer area 6021 and the audio data stream in the audio buffer area 6022 respectively. When the data volume of the video data stream buffered in the video buffer area 6021 is greater than the second preset threshold, the video data stream of the stop request is placed, and when the data volume of the audio data stream buffered in the audio buffer area 6022 is greater than the second preset threshold At this time, the drop receiving module stops requesting the audio data stream.
视频缓存区6031和音频缓存区6032为媒体播放器的缓存区，其中，视频缓存区6031用于缓存从视频缓存区6021读取的视频数据流，音频缓存区6032用于缓存从视频缓存区6022读取音频数据流。当获取到投放接收模块发送的数据读取指令时，媒体播放器从第二缓存区602的视频缓存区6021和音频缓存区6022分别读取的视频数据流和音频数据流，并根据PTS对音频数据流和视频数据流进行同步，将同步后的音频数据流和视频数据流分别缓存于视频缓存区6031和音频缓存区6032。The video buffer area 6031 and the audio buffer area 6032 are buffer areas of the media player, wherein the video buffer area 6031 is used to buffer the video data stream read from the video buffer area 6021, and the audio buffer area 6032 is used to buffer the video buffer area 6022 Read audio data stream. When the data reading instruction sent by the delivery and receiving module is obtained, the media player reads the video data stream and the audio data stream from the video buffer area 6021 and the audio buffer area 6022 of the second buffer area 602, respectively, and controls The data stream and the video data stream are synchronized, and the synchronized audio data stream and video data stream are buffered in the video buffer area 6031 and the audio buffer area 6032, respectively.
本实施例中视频数据流和音频数据分别通过三级缓存，有效保证了视频数据流和音频数据的流畅性和完整性，从而使得流媒体数据的视频数据流和音频数据的同步播放。In this embodiment, the video data stream and the audio data respectively pass through the three-level cache, which effectively guarantees the smoothness and integrity of the video data stream and the audio data, so that the video data stream and the audio data of the streaming media data are played synchronously.
上面对本申请实施例提供的媒体处理方法进行了描述，下面对本申请实施例提供的装置进行描述。The media processing method provided by the embodiment of the present application is described above, and the device provided by the embodiment of the present application is described below.
请参考图7，图7为本申请实施例提供的服务器的一个示意性框图，如图7所示，该服务器包括：Please refer to FIG. 7, which is a schematic block diagram of a server provided by an embodiment of the present application. As shown in FIG. 7, the server includes:
获取单元701，用于获取媒体播放请求；The obtaining unit 701 is used to obtain a media playback request;
处理单元702，用于根据所述媒体播放请求获取媒体播放页面的流化数据、所述终端设备请求播放的媒体的媒体信息和流媒体数据，所述媒体信息为所述流媒体数据的配置信息；The processing unit 702 is configured to obtain streaming data of a media playback page, media information and streaming media data of the media requested to be played by the terminal device according to the media playback request, where the media information is configuration information of the streaming media data ;
发送单元703，用于向所述终端设备发送所述流化数据和所述媒体信息，所述媒体信息用于所述终端设备创建媒体播放器，所述流化数据用于所述终端设备显示所述媒体播放页面；The sending unit 703 is configured to send the streaming data and the media information to the terminal device, the media information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display The media playback page;
所述发送单元703还用于向所述终端设备发送所述流媒体数据。The sending unit 703 is further configured to send the streaming media data to the terminal device.
可选地，本实施中，所述处理单元702具体用于：Optionally, in this implementation, the processing unit 702 is specifically configured to:
加载所述媒体播放请求对应的所述媒体播放页面，对所述媒体播放页面进行流化得到所述媒体播放页面的流化数据；Loading the media playback page corresponding to the media playback request, and streaming the media playback page to obtain streaming data of the media playback page;
从所述媒体播放页面获取所述媒体信息和所述流媒体数据。Obtain the media information and the streaming media data from the media playback page.
可选地，本实施中，所述流媒体数据包括音频数据流和视频数据流，所述发送单元703具体用于分别向所述终端设备发送所述音频数据流和所述视频数据流。Optionally, in this implementation, the streaming media data includes an audio data stream and a video data stream, and the sending unit 703 is specifically configured to send the audio data stream and the video data stream to the terminal device, respectively.
可选地，本实施中，所述获取单元701还用于从终端设备获取所述终端设备的解码器信息，所述解码器信息包括所述终端设备支持的解码格式；Optionally, in this implementation, the obtaining unit 701 is further configured to obtain decoder information of the terminal device from a terminal device, where the decoder information includes a decoding format supported by the terminal device;
所述发送单元703具体用于向所述终端设备发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。The sending unit 703 is specifically configured to send the streaming media data whose encoding format is a target encoding format to the terminal device, where the target encoding format matches the decoder information.
可选地，本实施中，所述发送单元703还用于向所述终端设备发送数字版权管理DRM证书，所述DRM证书用于所述终端设备对加密的所述流媒体数据进行解密。Optionally, in this implementation, the sending unit 703 is further configured to send a digital rights management DRM certificate to the terminal device, and the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
可选地，本实施中，所述获取单元701还用于从所述终端设备获取连接请求，所述连接请求包括所述终端设备的设备信息；Optionally, in this implementation, the obtaining unit 701 is further configured to obtain a connection request from the terminal device, where the connection request includes device information of the terminal device;
所述服务器还包括触发单元704和建立单元705，所述触发单元704用于根据所述设备信息触发设备管理平台对所述终端设备进行验证；所述建立单元705用于当所述设备管理平台对所述终端设备验证通过时，与所述终端设备建立连接关系。The server further includes a triggering unit 704 and a establishing unit 705, the triggering unit 704 is used to trigger a device management platform to verify the terminal device according to the device information; the establishing unit 705 is used to act as the device management platform When verification of the terminal device is passed, a connection relationship is established with the terminal device.
可选地，本实施中，所述服务器还包括第一缓存单元706，所述第一缓存单元706用于缓存所述流媒体数据于第一缓存区；Optionally, in this implementation, the server further includes a first cache unit 706, and the first cache unit 706 is configured to cache the streaming media data in the first cache area;
当缓存于所述第一缓存区的所述流媒体数据的数据量达到第一预置阈值时，所述发送单元703还用于向所述终端设备发送接收指令，所述接收指令用于指示所述终端设备接收所述流媒体数据。When the data volume of the streaming media data buffered in the first buffer area reaches a first preset threshold, the sending unit 703 is further configured to send a receiving instruction to the terminal device, and the receiving instruction is used to indicate The terminal device receives the streaming media data.
可选地，本实施中，所述获取单元701还用于接收所述终端设备发送的数据获取请求，所述数据获取请求包括请求数据的数据类型；Optionally, in this implementation, the acquisition unit 701 is further configured to receive a data acquisition request sent by the terminal device, where the data acquisition request includes the data type of the requested data;
所述发送单元703具体用于根据所述数据类型向所述终端设备发送缓存的所述流媒体数据。The sending unit 703 is specifically configured to send the buffered streaming media data to the terminal device according to the data type.
本实施例中，处理单元702根据终端设备发送的媒体播放请求获取媒体播放页面的流化数据和流媒体数据，发送单元703将流化数据和流媒体数据分别向终端设备发送，由终端设备根据流化数据显示媒体播放页面，以及由终端设备对流媒体数据进行解码播放。因此，本实施例中由终端设备对流媒体数据进行解码播放，降低了服务器资源的消耗，发送单元703将流化数据和流媒体数据分别向终端设备发送，无须对流化数据和流媒体数据进行重编码，降低了播控时延，提高了用户体验。In this embodiment, the processing unit 702 obtains the streaming data and streaming media data of the media playback page according to the media playback request sent by the terminal device, and the sending unit 703 sends the streaming data and streaming media data to the terminal device respectively, and the terminal device The streaming data displays the media playback page, and the terminal device decodes and plays the streaming media data. Therefore, in this embodiment, the terminal device decodes and plays the streaming media data to reduce the consumption of server resources. The sending unit 703 sends the streaming data and the streaming media data to the terminal device separately, without re-streaming the streaming data and the streaming media data. Encoding reduces broadcast control delay and improves user experience.
上面对本申请实施例提供的服务器进行了描述，下面对本申请实施例提供的终端设备进行描述。The server provided by the embodiment of the present application is described above, and the terminal device provided by the embodiment of the present application is described below.
请参考图8，图8为本申请实施例提供的终端设备的一个示意性框图。如图8所示，该终端设备包括：Please refer to FIG. 8, which is a schematic block diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 8, the terminal device includes:
发送单元801，用于向服务器发送媒体播放请求；The sending unit 801 is used to send a media playback request to the server;
接收单元802，用于接收所述服务器发送的媒体信息、流媒体数据和媒体播放页面的流化数据，所述媒体信息为所述流媒体数据的配置信息；The receiving unit 802 is configured to receive media information, streaming media data, and streaming data of a media playback page sent by the server, where the media information is configuration information of the streaming media data;
创建单元803，用于基于所述媒体信息创建媒体播放器；A creating unit 803, configured to create a media player based on the media information;
显示单元804，用于基于所述流化数据显示所述媒体播放页面；A display unit 804, configured to display the media playback page based on the streaming data;
播放单元805，用于在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据。The playing unit 805 is configured to play the streaming media data on the media playing page using the media player.
可选地，本实施例中，所述流媒体数据包括音频数据流和视频数据流，所述播放单元805具体用于在所述媒体播放页面上使用所述媒体播放器播放所述音频数据流和所述视频数据流；Optionally, in this embodiment, the streaming media data includes an audio data stream and a video data stream, and the playback unit 805 is specifically configured to play the audio data stream using the media player on the media playback page And the video data stream;
所述终端设备还包括同步单元806，所述同步单元806具体用于根据音视频帧的显示时间PTS对所述音频数据流和所述视频数据进行同步。The terminal device further includes a synchronization unit 806, which is specifically configured to synchronize the audio data stream and the video data according to the display time PTS of the audio and video frames.
可选地，本实施例中，所述接收单元802还用于接收所述服务器发送的数字版权管理DRM证书；Optionally, in this embodiment, the receiving unit 802 is further configured to receive a digital rights management DRM certificate sent by the server;
所述终端设备还包括解密单元807，所述解密单元用于根据所述DRM证书对所述流媒体数据进行解密。The terminal device further includes a decryption unit 807 for decrypting the streaming media data according to the DRM certificate.
可选地，本实施例中，所述接收单元802还用于接收所述服务器发送的接收指令；Optionally, in this embodiment, the receiving unit 802 is further configured to receive a receiving instruction sent by the server;
所述发送单元801还用于根据所述接收指令向所述服务器发送数据获取请求，所述数据获取请求包括请求数据的数据类型，所述数据类型用于指示所述服务器发送所述数据类型对应的流媒体数据。The sending unit 801 is further configured to send a data acquisition request to the server according to the receiving instruction, the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send the data type corresponding to Streaming data.
可选地，本实施例中，所述终端设备还包括第二缓存单元808，所述第二缓存单元808用于缓存所述流媒体数据于第二缓存区；Optionally, in this embodiment, the terminal device further includes a second cache unit 808, and the second cache unit 808 is configured to cache the streaming media data in the second cache area;
所述接收单元802还用于当缓存于所述第二缓存区所述流媒体数据的数据量达到第二预置阈值时，停止接收所述流媒体数据。The receiving unit 802 is further configured to stop receiving the streaming media data when the data volume of the streaming media data buffered in the second buffer area reaches a second preset threshold.
可选地，本实施例中，所述发送单元801还用于向所述服务器发送所述终端设备的解码器信息，所述解码器信息用于指示所述服务器发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。Optionally, in this embodiment, the sending unit 801 is further configured to send decoder information of the terminal device to the server, where the decoder information is used to instruct the server to send the encoding format to the target encoding format For the streaming media data, the target encoding format matches the decoder information.
可选地，本实施例中，所述发送单元801还用于向所述服务器发送连接请求，所述连接请求包括所述终端设备的设备信息，所述设备信息用于所述服务器对所述终端设备进行验证。Optionally, in this embodiment, the sending unit 801 is further configured to send a connection request to the server, where the connection request includes device information of the terminal device, and the device information is used by the server to End device verification.
本实施中，接收单元802分别接收所述服务器发送流媒体数据和媒体播放页面的流化数据，并由显示单元804根据流化数据进行显示媒体播放页面，以及由播放单元805对流媒体数据进行解码播放，降低了播放媒体的播控时延。In this implementation, the receiving unit 802 receives the streaming media data and the streaming data of the media playback page sent by the server respectively, and the display unit 804 displays the media playback page according to the streaming data, and the playback unit 805 decodes the streaming media data Play, reduce the broadcast control delay of playing media.
请参考图9，图9为本申请实施例提供的服务器的另一个示意性框图，如图9所示，该服务器包括浏览器201和投放模块202。其中，浏览器201执行的动作与前述图4对应实施例所涉及的浏览器所执行的动作类似，此处不再赘述，具体可参考图4。投放模块202执行的动作与前述图4对应实施例所涉及的投放模块所执行的动作类似，此处不再赘述，具体可参考图4。Please refer to FIG. 9, which is another schematic block diagram of a server provided by an embodiment of the present application. As shown in FIG. 9, the server includes a browser 201 and a delivery module 202. The actions performed by the browser 201 are similar to the actions performed by the browser involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again. For details, refer to FIG. 4. The actions performed by the delivery module 202 are similar to the actions performed by the delivery module involved in the embodiment corresponding to FIG. 4 described above, and details are not described herein again. For details, refer to FIG. 4.
本实施例提供的浏览器201和投放模块202的功能结构与前述图2对应实施例所描述的类似，此处不再赘述。The functional structures of the browser 201 and the delivery module 202 provided in this embodiment are similar to those described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
请参考图10，图10为本申请实施例提供的终端设备的另一个示意性框图，如图10所示，该终端设备包括投放接收模块203和媒体播放器204。其中，投放接收模块203执行的动作与前述图4对应实施例所涉及的投放接收模块所执行的动作类似，此处不再赘述，具体可参考图4。媒体播放器204执行的动作与前述图4对应实施例所涉及的媒体播放器所执行的动作类似，此处不再赘述，具体可参考图4。Please refer to FIG. 10, which is another schematic block diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 10, the terminal device includes a drop receiving module 203 and a media player 204. Wherein, the action performed by the delivery and receiving module 203 is similar to the action performed by the delivery and receiving module involved in the embodiment corresponding to FIG. 4 described above, which will not be repeated here. For details, refer to FIG. 4. The actions performed by the media player 204 are similar to the actions performed by the media player involved in the embodiment corresponding to FIG. 4 described above, and details are not described here again. For details, refer to FIG. 4.
本实施例提供的投放接收模块203和媒体播放器204的功能结构与前述图2对应实施例所描述的类似，此处不再赘述。The functional structure of the delivery receiving module 203 and the media player 204 provided in this embodiment is similar to that described in the corresponding embodiment of FIG. 2 described above, and details are not described herein again.
请参考图11，图11为本申请实施例提供的服务器的一个硬件结构示意图，如图11所示，该服务器包括：Please refer to FIG. 11, which is a schematic diagram of a hardware structure of a server provided by an embodiment of the present application. As shown in FIG. 11, the server includes:
至少一个处理器1110、存储器1150和收发器1130。该收发器可包括接收机和发射机，该存储器1150可以包括只读存储器和/或随机存取存储器，并向处理器1110提供操作指令和数据。存储器1150的一部分还可以包括非易失性随机存取存储器(NVRAM)。存储器与处理器可以是各自独立通过总线或者接口连接，也可以集成在一起。At least one processor 1110, memory 1150, and transceiver 1130. The transceiver may include a receiver and a transmitter. The memory 1150 may include a read-only memory and / or a random access memory, and provide operation instructions and data to the processor 1110. A portion of the memory 1150 may also include non-volatile random access memory (NVRAM). The memory and the processor may be independently connected through a bus or an interface, or may be integrated together.
在一些实施方式中，存储器1150存储了如下的元素，可执行模块或者数据结构，或者他们的子集，或者他们的扩展集。In some embodiments, the memory 1150 stores the following elements, executable modules or data structures, or their subsets, or their extensions.
在本申请实施例中，通过调用存储器1150存储的操作指令(该操作指令可存储在操作***中)，执行相应的操作。处理器1110控制服务器的操作，处理器1110还可以称为CPU(Central Processing Unit，中央处理单元)。存储器1150可以包括只读存储器和随机存取存储器，并向处理器1110提供指令和数据。存储器1150的一部分还可以包括非易失性随机存取存储器(NVRAM)。具体的应用中服务器的各个组件通过总线***1120耦合在一起，其中总线***1120除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线***1120。In the embodiment of the present application, the corresponding operation is performed by calling the operation instruction stored in the memory 1150 (the operation instruction may be stored in the operating system). The processor 1110 controls the operation of the server, and the processor 1110 may also be called a CPU (Central Processing Unit, central processing unit). The memory 1150 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1110. A portion of the memory 1150 may also include non-volatile random access memory (NVRAM). In a specific application, the various components of the server are coupled together through a bus system 1120. In addition to the data bus, the bus system 1120 may also include a power bus, a control bus, and a status signal bus. However, for clarity, various buses are marked as the bus system 1120 in the figure.
上述本申请实施例揭示的方法可以应用于处理器1110中，或者由处理器1110实现。处理器1110可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器1110中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1110可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1150，该存储器1150可以是物理上独立的单元，也可以是与处理器1110集成在一起的，处理器1110读取存储器1150中的信息，结合其硬件完成上述方法的步骤。The method disclosed in the above embodiments of the present application may be applied to the processor 1110 or implemented by the processor 1110. The processor 1110 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 1110 or an instruction in the form of software. The aforementioned processor 1110 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware Components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied and executed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register. The storage medium is located in the memory 1150. The memory 1150 may be a physically independent unit, or may be integrated with the processor 1110. The processor 1110 reads the information in the memory 1150 and completes the steps of the above method in combination with its hardware.
本实施例中，收发器1130可以分别用于执行图3对应的实施例以及图4对应的实施例服务器侧涉及到接收和发送的操作步骤。或用于执行其他可选实施例中的服务器侧的数据发送以及接收的步骤。In this embodiment, the transceiver 1130 may be used to execute the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4 on the server side related to the receiving and sending operation steps, respectively. Or used to perform the steps of sending and receiving data on the server side in other optional embodiments.
处理器1110可以分别用于执行图3对应的实施例以及图4对应的实施例服务器侧数据处理的步骤。或用于执行其他可选实施例中服务器侧数据处理的步骤。The processor 1110 may be used to execute the steps of server-side data processing in the embodiment corresponding to FIG. 3 and the embodiment corresponding to FIG. 4, respectively. Or it is used to perform the steps of server-side data processing in other optional embodiments.
请参考图12，图12为本申请实施例提供的终端设备的一个硬件结构示意图，图12示出了一种简化的终端设备的硬件结构示意图。便于理解和图示方便，图12中，终端设备以手机作为例子。如图12所示，终端设备包括处理器、存储器、射频电路、天线以及输入输出装置。处理器主要用于对通信协议以及通信数据进行处理，以及对终端设备进行控制，执行软件程序，处理软件程序的数据等。存储器主要用于存储软件程序和数据。射频电路主要用于基带信号与射频信号的转换以及对射频信号的处理。天线主要用于收发电磁波形式的射频信号。输入输出装置，例如触摸屏、显示屏，键盘等主要用于接收用户输入的数据以及对用户输出数据。需要说明的是，有些种类的终端设备可以不具有输入输出装置。Please refer to FIG. 12, which is a schematic diagram of a hardware structure of a terminal device provided by an embodiment of the present application. FIG. 12 shows a simplified schematic diagram of a hardware structure of a terminal device. It is easy to understand and convenient to illustrate. In FIG. 12, the terminal device uses a mobile phone as an example. As shown in FIG. 12, the terminal device includes a processor, a memory, a radio frequency circuit, an antenna, and input and output devices. The processor is mainly used for processing communication protocols and communication data, as well as controlling terminal devices, executing software programs, and processing data of software programs. The memory is mainly used to store software programs and data. The radio frequency circuit is mainly used for the conversion of the baseband signal and the radio frequency signal and the processing of the radio frequency signal. The antenna is mainly used to send and receive radio frequency signals in the form of electromagnetic waves. Input and output devices, such as touch screens, display screens, and keyboards, are mainly used to receive data input by users and output data to users. It should be noted that some types of terminal devices may not have input / output devices.
当需要发送数据时，处理器对待发送的数据进行基带处理后，输出基带信号至射频电路，射频电路将基带信号进行射频处理后将射频信号通过天线以电磁波的形式向外发送。当有数据发送到终端设备时，射频电路通过天线接收到射频信号，将射频信号转换为基带信号，并将基带信号输出至处理器，处理器将基带信号转换为数据并对该数据进行处理。为便于说明，图12中仅示出了一个存储器和处理器。在实际的终端设备产品中，可以存在一个或多个处理器和一个或多个存储器。存储器也可以称为存储介质或者存储设备等。存 储器可以是独立于处理器设置，也可以是与处理器集成在一起，本申请实施例对此不做限制。When data needs to be sent, the processor performs baseband processing on the data to be sent, and then outputs the baseband signal to the radio frequency circuit. The radio frequency circuit processes the baseband signal after radio frequency processing and sends the radio frequency signal to the outside in the form of electromagnetic waves through the antenna. When data is sent to the terminal device, the radio frequency circuit receives the radio frequency signal through the antenna, converts the radio frequency signal into a baseband signal, and outputs the baseband signal to the processor. The processor converts the baseband signal into data and processes the data. For ease of explanation, only one memory and processor are shown in FIG. 12. In actual terminal equipment products, there may be one or more processors and one or more memories. The memory may also be referred to as a storage medium or storage device. The memory may be set independently of the processor, or may be integrated with the processor, which is not limited in the embodiments of the present application.
在本申请实施例中，可以将具有收发功能的天线和射频电路视为终端设备的收发单元，将具有处理功能的处理器视为终端设备的处理单元。其中。收发单元也可以称为收发器、收发机、收发装置等。处理单元也可以称为处理器，处理单板，处理模块、处理装置等。可选的，可以将收发单元中用于实现接收功能的器件视为接收单元，将收发单元中用于实现发送功能的器件视为发送单元，即收发单元包括接收单元和发送单元。收发单元有时也可以称为收发机、收发器、或收发电路等。接收单元有时也可以称为接收机、接收器、或接收电路等。发送单元有时也可以称为发射机、发射器或者发射电路等。In the embodiments of the present application, an antenna and a radio frequency circuit with a transceiver function can be regarded as a transceiver unit of a terminal device, and a processor with a processing function can be regarded as a processing unit of the terminal device. among them. The transceiver unit may also be called a transceiver, a transceiver, a transceiver device, or the like. The processing unit may also be called a processor, a processing board, a processing module, a processing device, and the like. Optionally, the device used to implement the receiving function in the transceiver unit can be regarded as the receiving unit, and the device used to implement the sending function in the transceiver unit can be regarded as the sending unit, that is, the transceiver unit includes a receiving unit and a sending unit. The transceiver unit may sometimes be called a transceiver, transceiver, or transceiver circuit. The receiving unit may sometimes be called a receiver, a receiver, or a receiving circuit. The sending unit may sometimes be called a transmitter, a transmitter, or a transmitting circuit.
可以理解的是，收发单元用于执行上述方法实施例中终端设备侧所设备的发送操作和接收操作，处理单元用于执行上述方法实施例中终端设备上除了收发操作之外的其他处理操作。It can be understood that the transceiving unit is used to perform the sending operation and the receiving operation on the terminal device side in the above method embodiment, and the processing unit is used to perform other processing operations on the terminal device in addition to the transceiving operation in the above method embodiment.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that for the convenience and conciseness of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露的***，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The above integrated unit can be implemented in the form of hardware or software function unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application may be essentially or part of the contribution to the existing technology or all or part of the technical solution may be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they can still The technical solutions described in the embodiments are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate from the scope of the technical solutions of the embodiments of the present application.

Claims (32)

  1. 一种媒体处理方法，其特征在于，包括：A media processing method, which includes:
    服务器从终端设备获取媒体播放请求；The server obtains the media playback request from the terminal device;
    所述服务器根据所述媒体播放请求获取媒体播放页面的流化数据、所述终端设备请求播放的媒体的媒体信息和流媒体数据，所述媒体信息为所述流媒体数据的配置信息；The server obtains streaming data of a media playback page, media information and streaming media data of the media requested to be played by the terminal device according to the media playback request, and the media information is configuration information of the streaming media data;
    所述服务器向所述终端设备发送所述流化数据和所述媒体信息，所述媒体信息用于所述终端设备创建媒体播放器，所述流化数据用于所述终端设备显示所述媒体播放页面；The server sends the streaming data and the media information to the terminal device, the media information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display the media Play page
    所述服务器向所述终端设备发送所述流媒体数据。The server sends the streaming media data to the terminal device.
  2. 根据权利要求1所述的方法，其特征在于，所述服务器根据所述媒体播放请求获取媒体播放页面的流化数据、所述终端设备请求播放的媒体的媒体信息和流媒体数据包括：The method according to claim 1, wherein the server acquiring streaming data of the media playback page, media information and streaming media data of the media requested by the terminal device according to the media playback request includes:
    所述服务器加载所述媒体播放请求对应的所述媒体播放页面，对所述媒体播放页面进行流化得到所述媒体播放页面的流化数据；The server loads the media playback page corresponding to the media playback request, and streams the media playback page to obtain streaming data of the media playback page;
    所述服务器从所述媒体播放页面获取所述媒体信息和所述流媒体数据。The server obtains the media information and the streaming media data from the media playback page.
  3. 根据权利要求1或2所述的方法，其特征在于，所述流媒体数据包括音频数据流和视频数据流，所述服务器向所述终端设备发送所述流媒体数据包括：The method according to claim 1 or 2, wherein the streaming media data includes an audio data stream and a video data stream, and the server sending the streaming media data to the terminal device includes:
    所述服务器分别向所述终端设备发送所述音频数据流和所述视频数据流。The server sends the audio data stream and the video data stream to the terminal device, respectively.
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述服务器向所述终端设备发送所述流媒体数据之前，所述方法还包括：The method according to any one of claims 1 to 3, wherein before the server sends the streaming media data to the terminal device, the method further comprises:
    所述服务器从终端设备获取所述终端设备的解码器信息，所述解码器信息包括所述终端设备支持的解码格式；The server obtains decoder information of the terminal device from the terminal device, and the decoder information includes a decoding format supported by the terminal device;
    所述服务器向所述终端设备发送所述流媒体数据包括：The server sending the streaming media data to the terminal device includes:
    所述服务器向所述终端设备发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。The server sends the streaming media data whose encoding format is a target encoding format to the terminal device, and the target encoding format matches the decoder information.
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述流媒体数据为加密流，所述方法还包括：The method according to any one of claims 1 to 4, wherein the streaming media data is an encrypted stream, and the method further includes:
    所述服务器向所述终端设备发送数字版权管理DRM证书，所述DRM证书用于所述终端设备对加密的所述流媒体数据进行解密。The server sends a digital rights management DRM certificate to the terminal device, and the DRM certificate is used by the terminal device to decrypt the encrypted streaming media data.
  6. 根据权利要求5所述的方法，其特征在于，所述服务器从终端设备获取所述终端设备的解码器信息之前，所述方法还包括：The method according to claim 5, wherein before the server obtains the decoder information of the terminal device from the terminal device, the method further comprises:
    所述服务器从所述终端设备获取连接请求，所述连接请求包括所述终端设备的设备信息；The server obtains a connection request from the terminal device, where the connection request includes device information of the terminal device;
    所述服务器根据所述设备信息触发设备管理平台对所述终端设备进行验证；The server triggers a device management platform to verify the terminal device according to the device information;
    当所述设备管理平台对所述终端设备验证通过时，所述服务器与所述终端设备建立连接关系。When the device management platform passes the verification of the terminal device, the server establishes a connection relationship with the terminal device.
  7. 根据权利要求1-6任一项所述的方法，其特征在于，所述服务器向所述终端设备发送所述流媒体数据之前，所述方法还包括：The method according to any one of claims 1-6, wherein before the server sends the streaming media data to the terminal device, the method further comprises:
    所述服务器缓存所述流媒体数据于第一缓存区；The server caches the streaming media data in the first cache area;
    当缓存于所述第一缓存区的所述流媒体数据的数据量达到第一预置阈值时，所述服务 器向所述终端设备发送接收指令，所述接收指令用于指示所述终端设备接收所述流媒体数据。When the data volume of the streaming media data cached in the first cache area reaches a first preset threshold, the server sends a reception instruction to the terminal device, and the reception instruction is used to instruct the terminal device to receive The streaming media data.
  8. 根据权利要求7所述的方法，其特征在于，所述服务器向所述终端设备发送所述流媒体数据包括：The method according to claim 7, wherein the server sending the streaming media data to the terminal device comprises:
    所述服务器接收所述终端设备发送的数据获取请求，所述数据获取请求包括请求数据的数据类型；The server receives a data acquisition request sent by the terminal device, where the data acquisition request includes the data type of the requested data;
    所述服务器根据所述数据类型向所述终端设备发送缓存的所述流媒体数据。The server sends the buffered streaming media data to the terminal device according to the data type.
  9. 一种媒体处理方法，其特征在于，包括：A media processing method, which includes:
    终端设备向服务器发送媒体播放请求；The terminal device sends a media playback request to the server;
    所述终端设备接收所述服务器发送的媒体信息、流媒体数据和媒体播放页面的流化数据，所述媒体信息为所述流媒体数据的配置信息；The terminal device receives media information, streaming media data, and streaming data of a media playback page sent by the server, where the media information is configuration information of the streaming media data;
    所述终端设备基于所述媒体信息创建媒体播放器；The terminal device creates a media player based on the media information;
    所述终端设备基于所述流化数据显示所述媒体播放页面；The terminal device displays the media playback page based on the streaming data;
    所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据。The terminal device uses the media player to play the streaming media data on the media playing page.
  10. 根据权利要求9所述的方法，其特征在于，所述流媒体数据包括音频数据流和视频数据流，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据包括：The method according to claim 9, wherein the streaming media data includes an audio data stream and a video data stream, and the terminal device uses the media player to play the streaming media data on the media playing page include:
    所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述音频数据流和所述视频数据流；The terminal device uses the media player to play the audio data stream and the video data stream on the media playing page;
    所述终端设备根据音视频帧的显示时间PTS对所述音频数据流和所述视频数据进行同步。The terminal device synchronizes the audio data stream and the video data according to the display time PTS of the audio and video frames.
  11. 根据权利要求9或10所述的方法，其特征在于，所述终端设备在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据之前，所述方法还包括：The method according to claim 9 or 10, wherein before the terminal device uses the media player to play the streaming media data on the media playing page, the method further comprises:
    所述终端设备接收所述服务器发送的数字版权管理DRM证书；The terminal device receives the digital rights management DRM certificate sent by the server;
    所述终端设备根据所述DRM证书对所述流媒体数据进行解密。The terminal device decrypts the streaming media data according to the DRM certificate.
  12. 根据权利要求9-11任一项所述的方法，其特征在于，终端设备接收所述服务器发送的所述流媒体数据之前，所述方法还包括：The method according to any one of claims 9-11, wherein before the terminal device receives the streaming media data sent by the server, the method further comprises:
    所述终端设备接收所述服务器发送的接收指令；The terminal device receives the receiving instruction sent by the server;
    所述终端设备根据所述接收指令向所述服务器发送数据获取请求，所述数据获取请求包括请求数据的数据类型，所述数据类型用于指示所述服务器发送所述数据类型对应的流媒体数据。The terminal device sends a data acquisition request to the server according to the receiving instruction, where the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send streaming media data corresponding to the data type .
  13. 根据权利要求9-12任一项所述的方法，其特征在于，终端设备接收所述服务器发送的所述流媒体数据之后，所述方法还包括：The method according to any one of claims 9-12, wherein after the terminal device receives the streaming media data sent by the server, the method further comprises:
    所述终端设备缓存所述流媒体数据于第二缓存区；The terminal device caches the streaming media data in the second cache area;
    当缓存于所述第二缓存区所述流媒体数据的数据量达到第二预置阈值时，所述终端设备停止接收所述流媒体数据。When the data amount of the streaming media data buffered in the second buffer area reaches a second preset threshold, the terminal device stops receiving the streaming media data.
  14. 根据权利要求9-13任一项所述的方法，其特征在于，所述终端设备向服务器发送媒体播放请求之前，所述方法还包括：The method according to any one of claims 9-13, wherein before the terminal device sends a media playback request to the server, the method further comprises:
    所述终端设备向所述服务器发送所述终端设备的解码器信息，所述解码器信息用于指示所述服务器发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。The terminal device sends decoder information of the terminal device to the server, where the decoder information is used to instruct the server to send the streaming media data whose encoding format is a target encoding format, and the target encoding format is The decoder information matches.
  15. 根据权利要求14所述的方法，其特征在于，所述终端设备向所述服务器发送所述终端设备的解码器信息之前，所述方法还包括：The method according to claim 14, wherein before the terminal device sends decoder information of the terminal device to the server, the method further comprises:
    所述终端设备向所述服务器发送连接请求，所述连接请求包括所述终端设备的设备信息，所述设备信息用于所述服务器对所述终端设备进行验证。The terminal device sends a connection request to the server, where the connection request includes device information of the terminal device, and the device information is used by the server to verify the terminal device.
  16. 一种服务器，其特征在于，包括：A server, characterized in that it includes:
    获取单元，用于获取媒体播放请求；An obtaining unit, used to obtain a media playing request;
    处理单元，用于根据所述媒体播放请求获取媒体播放页面的流化数据、所述终端设备请求播放的媒体的媒体信息和流媒体数据，所述媒体信息为所述流媒体数据的配置信息；A processing unit, configured to obtain streaming data of a media playback page, media information and streaming media data of the media requested to be played by the terminal device according to the media playback request, where the media information is configuration information of the streaming media data;
    发送单元，用于向所述终端设备发送所述流化数据和所述媒体信息，所述媒体信息用于所述终端设备创建媒体播放器，所述流化数据用于所述终端设备显示所述媒体播放页面；A sending unit, configured to send the streaming data and the media information to the terminal device, the media information is used by the terminal device to create a media player, and the streaming data is used by the terminal device to display Describe the media playback page;
    所述发送单元还用于向所述终端设备发送所述流媒体数据。The sending unit is also used to send the streaming media data to the terminal device.
  17. 根据权利要求16所述的服务器，其特征在于，所述处理单元具体用于：The server according to claim 16, wherein the processing unit is specifically configured to:
    加载所述媒体播放请求对应的所述媒体播放页面，对所述媒体播放页面进行流化得到所述媒体播放页面的流化数据；Loading the media playback page corresponding to the media playback request, and streaming the media playback page to obtain streaming data of the media playback page;
    从所述媒体播放页面获取所述媒体信息和所述流媒体数据。Obtain the media information and the streaming media data from the media playback page.
  18. 根据权利要求16或17所述的服务器，其特征在于，所述流媒体数据包括音频数据流和视频数据流，所述发送单元具体用于分别向所述终端设备发送所述音频数据流和所述视频数据流。The server according to claim 16 or 17, wherein the streaming media data includes an audio data stream and a video data stream, and the sending unit is specifically configured to send the audio data stream and the data stream to the terminal device, respectively. Describe the video data stream.
  19. 根据权利要求16-18任一项所述的服务器，其特征在于，所述获取单元还用于从终端设备获取所述终端设备的解码器信息，所述解码器信息包括所述终端设备支持的解码格式；The server according to any one of claims 16 to 18, wherein the acquiring unit is further configured to acquire decoder information of the terminal device from a terminal device, and the decoder information includes information supported by the terminal device. Decoding format
    所述发送单元具体用于向所述终端设备发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。The sending unit is specifically configured to send the streaming media data whose encoding format is a target encoding format to the terminal device, where the target encoding format matches the decoder information.
  20. 根据权利要求16-19任一项所述的服务器，其特征在于，所述发送单元还用于向所述终端设备发送数字版权管理DRM证书，所述DRM证书用于所述终端设备对加密的所述流媒体数据进行解密。The server according to any one of claims 16 to 19, wherein the sending unit is further configured to send a digital rights management DRM certificate to the terminal device, and the DRM certificate is used by the terminal device to encrypt The streaming media data is decrypted.
  21. 根据权利要求20所述的服务器，其特征在于，所述获取单元还用于从所述终端设备获取连接请求，所述连接请求包括所述终端设备的设备信息；The server according to claim 20, wherein the acquiring unit is further configured to acquire a connection request from the terminal device, the connection request including device information of the terminal device;
    所述服务器还包括触发单元和建立单元，所述触发单元用于根据所述设备信息触发设备管理平台对所述终端设备进行验证；所述建立单元用于当所述设备管理平台对所述终端设备验证通过时，与所述终端设备建立连接关系。The server further includes a triggering unit and a establishing unit, the triggering unit is configured to trigger a device management platform to verify the terminal device based on the device information; the establishing unit is configured to use the device management platform to authenticate the terminal When the device verification is passed, a connection relationship is established with the terminal device.
  22. 根据权利要求16-21任一项所述的服务器，其特征在于，所述服务器还包括第一缓存单元，所述第一缓存单元用于缓存所述流媒体数据于第一缓存区；The server according to any one of claims 16 to 21, wherein the server further comprises a first cache unit, and the first cache unit is used to cache the streaming media data in the first cache area;
    当缓存于所述第一缓存区的所述流媒体数据的数据量达到第一预置阈值时，所述发送单元还用于向所述终端设备发送接收指令，所述接收指令用于指示所述终端设备接收所述 流媒体数据。When the data volume of the streaming media data buffered in the first buffer area reaches a first preset threshold, the sending unit is further used to send a receiving instruction to the terminal device, the receiving instruction is used to indicate The terminal device receives the streaming media data.
  23. 根据权利要求22所述的服务器，其特征在于，所述获取单元还用于接收所述终端设备发送的数据获取请求，所述数据获取请求包括请求数据的数据类型；The server according to claim 22, wherein the acquisition unit is further configured to receive a data acquisition request sent by the terminal device, the data acquisition request including a data type of the requested data;
    所述发送单元具体用于根据所述数据类型向所述终端设备发送缓存的所述流媒体数据。The sending unit is specifically configured to send the buffered streaming media data to the terminal device according to the data type.
  24. 一种终端设备，其特征在于，包括：A terminal device is characterized by comprising:
    发送单元，用于向服务器发送媒体播放请求；The sending unit is used to send a media playback request to the server;
    接收单元，用于接收所述服务器发送的媒体信息、流媒体数据和媒体播放页面的流化数据，所述媒体信息为所述流媒体数据的配置信息；A receiving unit, configured to receive media information, streaming media data, and streaming data of a media playback page sent by the server, where the media information is configuration information of the streaming media data;
    创建单元，用于基于所述媒体信息创建媒体播放器；Creating unit for creating a media player based on the media information;
    显示单元，用于基于所述流化数据显示所述媒体播放页面；A display unit, configured to display the media playback page based on the streaming data;
    播放单元，用于在所述媒体播放页面上使用所述媒体播放器播放所述流媒体数据。The playing unit is configured to play the streaming media data on the media playing page using the media player.
  25. 根据权利要求24所述的终端设备，其特征在于，所述流媒体数据包括音频数据流和视频数据流，所述播放单元具体用于在所述媒体播放页面上使用所述媒体播放器播放所述音频数据流和所述视频数据流；The terminal device according to claim 24, wherein the streaming media data includes an audio data stream and a video data stream, and the playback unit is specifically configured to use the media player to play the media player on the media playback page. The audio data stream and the video data stream;
    所述终端设备还包括同步单元，所述同步单元具体用于根据音视频帧的显示时间PTS对所述音频数据流和所述视频数据进行同步。The terminal device further includes a synchronization unit, which is specifically configured to synchronize the audio data stream and the video data according to the display time PTS of the audio and video frames.
  26. 根据权利要求24或25所述的终端设备，其特征在于，所述接收单元还用于接收所述服务器发送的数字版权管理DRM证书；The terminal device according to claim 24 or 25, wherein the receiving unit is further configured to receive a digital rights management DRM certificate sent by the server;
    所述终端设备还包括解密单元，所述解密单元用于根据所述DRM证书对所述流媒体数据进行解密。The terminal device further includes a decryption unit for decrypting the streaming media data according to the DRM certificate.
  27. 根据权利要求24-26任一项所述的终端设备，其特征在于，所述接收单元还用于接收所述服务器发送的接收指令；The terminal device according to any one of claims 24-26, wherein the receiving unit is further configured to receive a receiving instruction sent by the server;
    所述发送单元还用于根据所述接收指令向所述服务器发送数据获取请求，所述数据获取请求包括请求数据的数据类型，所述数据类型用于指示所述服务器发送所述数据类型对应的流媒体数据。The sending unit is further configured to send a data acquisition request to the server according to the receiving instruction, the data acquisition request includes a data type of the requested data, and the data type is used to instruct the server to send the data type corresponding to the data type. Streaming data.
  28. 根据权利要求24-27任一项所述的终端设备，其特征在于，所述终端设备还包括第二缓存单元，所述第二缓存单元用于缓存所述流媒体数据于第二缓存区；The terminal device according to any one of claims 24-27, wherein the terminal device further comprises a second cache unit, and the second cache unit is used to cache the streaming media data in a second cache area;
    所述接收单元还用于当缓存于所述第二缓存区所述流媒体数据的数据量达到第二预置阈值时，停止接收所述流媒体数据。The receiving unit is further configured to stop receiving the streaming media data when the data volume of the streaming media data buffered in the second buffer area reaches a second preset threshold.
  29. 根据权利要求24-28任一项所述的终端设备，其特征在于，所述发送单元还用于向所述服务器发送所述终端设备的解码器信息，所述解码器信息用于指示所述服务器发送编码格式为目标编码格式的所述流媒体数据，所述目标编码格式与所述解码器信息匹配。The terminal device according to any one of claims 24-28, wherein the sending unit is further configured to send decoder information of the terminal device to the server, and the decoder information is used to indicate the The server sends the streaming media data whose encoding format is the target encoding format, and the target encoding format matches the decoder information.
  30. 根据权利要求29所述的终端设备，其特征在于，所述发送单元还用于向所述服务器发送连接请求，所述连接请求包括所述终端设备的设备信息，所述设备信息用于所述服务器对所述终端设备进行验证。The terminal device according to claim 29, wherein the sending unit is further configured to send a connection request to the server, where the connection request includes device information of the terminal device, and the device information is used for the The server verifies the terminal device.
  31. 一种计算机可读存储介质，其特征在于，包括指令，当所述指令在计算机上运行时，使得计算机设备执行权利要求1至8中任一项所述的方法。A computer-readable storage medium, comprising instructions, which when executed on a computer, causes a computer device to perform the method of any one of claims 1 to 8.
  32. 一种计算机可读存储介质，其特征在于，包括指令，当所述指令在计算机上运行时，使得计算机设备执行权利要求9至15中任一项所述的方法。A computer-readable storage medium, comprising instructions, which when executed on a computer, causes a computer device to perform the method according to any one of claims 9 to 15.
PCT/CN2019/106327 2018-10-12 2019-09-18 Media processing method and apparatus WO2020073777A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811191179.8A CN111050192A (en) 2018-10-12 2018-10-12 Media processing method and device
CN201811191179.8 2018-10-12

Publications (1)

Publication Number Publication Date
WO2020073777A1 true WO2020073777A1 (en) 2020-04-16

Family

ID=70164235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/106327 WO2020073777A1 (en) 2018-10-12 2019-09-18 Media processing method and apparatus

Country Status (2)

Country Link
CN (1) CN111050192A (en)
WO (1) WO2020073777A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113595976A (en) * 2021-06-15 2021-11-02 阿里巴巴新加坡控股有限公司 Multimedia playing method, cloud server, system and storage medium
CN115695843B (en) * 2022-10-27 2024-06-18 中国联合网络通信集团有限公司 Prefabricated video playing method, server, terminal, medium and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102215424A (en) * 2010-04-12 2011-10-12 腾讯科技（深圳）有限公司 Internet protocol television direct transmission method, device and system
CN103561072A (en) * 2013-10-25 2014-02-05 李俊松 Cloud data processing method based on Internet and cloud digital photo frame
CN104159150A (en) * 2014-08-19 2014-11-19 龙芯中科技术有限公司 Cloud terminal, cloud server, media data stream playing system and method
US20170006314A1 (en) * 2014-05-06 2017-01-05 Tivo Inc. Managing media content upload groups

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102281473A (en) * 2011-09-09 2011-12-14 青岛海信电器股份有限公司 Information processing method for television terminal and television terminal
CN103179431B (en) * 2013-03-21 2016-06-01 福建升腾资讯有限公司 Under VDI environment, audio frequency and video are redirected and turn code separation method
CN104168486A (en) * 2013-05-20 2014-11-26 上海范氏科技投资发展集团有限公司 Virtual machine and client video redirection method based on cloud computing
CN104301742A (en) * 2013-07-16 2015-01-21 上海国富光启云计算科技有限公司 Video redirecting device between virtual machine and client and use method thereof
CN105611394B (en) * 2015-12-21 2018-11-06 福建时迅信息科技有限公司 Video reorientation method under VDI environment and system
CN107360443B (en) * 2016-05-09 2019-10-01 南京中兴新软件有限责任公司 A kind of cloud desktop picture processing method, cloud desktop server and client
CN108174214A (en) * 2017-12-08 2018-06-15 重庆邮电大学 A kind of remote table sharing method based on screen content Video coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102215424A (en) * 2010-04-12 2011-10-12 腾讯科技（深圳）有限公司 Internet protocol television direct transmission method, device and system
CN103561072A (en) * 2013-10-25 2014-02-05 李俊松 Cloud data processing method based on Internet and cloud digital photo frame
US20170006314A1 (en) * 2014-05-06 2017-01-05 Tivo Inc. Managing media content upload groups
CN104159150A (en) * 2014-08-19 2014-11-19 龙芯中科技术有限公司 Cloud terminal, cloud server, media data stream playing system and method

Also Published As

Publication number Publication date
CN111050192A (en) 2020-04-21

Similar Documents

Publication Publication Date Title
US20220263885A1 (en) Adaptive media streaming method and apparatus according to decoding performance
US11368731B2 (en) Method and apparatus for segmenting data
US12010158B2 (en) Systems and methods for multi-device media broadcasting or recording with low-latency active control
JP2021514167A (en) Distribution and playback of media content
US8902967B2 (en) Systems and methods for distributed media stream transcoding and sharing
EP3267331B1 (en) Method and apparatus for cloud streaming service
CN107211186B (en) Method and apparatus for providing multi-view streaming service
US10171530B2 (en) Devices and methods for transmitting adaptively adjusted documents
EP3937434B1 (en) Data distribution method and network device
US9356820B2 (en) Maximizing throughput of streaming media by simultaneously connecting to streaming media server over multiple independent network connections
WO2021031739A1 (en) Cloud desktop video playback method, server, terminal, and storage medium
WO2012151957A1 (en) Server, client, and method and system for playing video file remotely therewith
WO2020073777A1 (en) Media processing method and apparatus
US20150086184A1 (en) Reception apparatus and reception method
KR102299615B1 (en) Method and apparatus for mpeg media transport integration in content distribution networks
US20160269757A1 (en) Apparatus, systems and methods for remote storage of media content events
CN113709493A (en) Video stream data encryption device, method and equipment of KVM system
WO2024060719A1 (en) Data transmission methods, apparatus, electronic device, and storage medium
US20120284764A1 (en) Method and system for requesting services by a media device
WO2024087208A1 (en) Video playback method and system, and storage medium
EP3785443A1 (en) System and method for rendering stitched video media stream on a display device
US20150156237A1 (en) Video Streaming System, Computer Device, and Video Streaming Method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19870798

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19870798

Country of ref document: EP

Kind code of ref document: A1