WO2022105798A1 - 音视频的处理方法、装置及存储介质 - Google Patents

音视频的处理方法、装置及存储介质 Download PDF

Info

Publication number
WO2022105798A1
WO2022105798A1 PCT/CN2021/131226 CN2021131226W WO2022105798A1 WO 2022105798 A1 WO2022105798 A1 WO 2022105798A1 CN 2021131226 W CN2021131226 W CN 2021131226W WO 2022105798 A1 WO2022105798 A1 WO 2022105798A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
audio
terminal device
server
network
Prior art date
Application number
PCT/CN2021/131226
Other languages
English (en)
French (fr)
Inventor
施小龙
曹振
Original Assignee
花瓣云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 花瓣云科技有限公司 filed Critical 花瓣云科技有限公司
Publication of WO2022105798A1 publication Critical patent/WO2022105798A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
    • H04L12/1827—Network arrangements for conference optimisation or adaptation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2387—Stream processing in response to a playback request from an end-user, e.g. for trick-play
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand

Definitions

  • the present application relates to the field of multimedia technologies, and in particular, to an audio and video processing method, device, and storage medium.
  • audio and video playback generally adopts a download mode based on audio and video segments.
  • the server divides versions of the same audio and video with different bit rates into fragments of preset lengths, and encapsulates each fragment.
  • the terminal device When the terminal device needs to play audio and video, it sends a connection request to the server. After receiving the connection request, the server performs bandwidth detection through multiple interactions with the terminal device to determine an appropriate initial window size for network communication. After receiving the audio and video playback request sent by the terminal device, the server transmits the initial fragmentation of the audio and video according to the determined initial window size of the network communication.
  • an audio and video processing method, device and storage medium are proposed.
  • a terminal device sends an audio and video playback request to a server, it actively reports the network quality used to indicate the current network quality of the terminal device's access network. information, avoids multiple interactions between the server and the terminal device for network bandwidth detection in the related art, thereby shortening the playback delay of audio and video, and ensuring the playback effect of audio and video.
  • an embodiment of the present application provides a method for processing audio and video, which is used in a terminal device, and the method includes:
  • an audio and video playback request carrying network quality information is sent to the server.
  • the network quality information is used to indicate the current network quality of the terminal device accessing the network, and the audio and video playback request is used to instruct the server to return the audio and video of the target audio and video.
  • the terminal device after receiving the target audio and video playback instruction, the terminal device sends an audio and video playback request carrying network quality information to the server, and the audio and video playback request is used to instruct the server to return the target audio and video audio and video data; the terminal The device downloads and plays the audio and video data of the target audio and video returned by the server; that is, when the terminal device sends an audio and video playback request to the server, it actively reports the network quality information of the current access network of the terminal device, avoiding the need for the server and the terminal in the related art. In the case of multiple interactions between devices for network bandwidth detection, the playback delay of audio and video is shortened and the playback effect of audio and video is guaranteed.
  • the network quality information includes the network bandwidth of the current access network of the terminal device.
  • the terminal device actively reports the network bandwidth of the current access network of the terminal device, which avoids the situation in the related art that the network bandwidth needs to be acquired by the server through multiple RTT detections.
  • the audio and video data includes at least one audio and video segment, and the method further includes:
  • ACK acknowledgment
  • the terminal device in order to prevent the ACK message from being lost, the terminal device sends multiple ACK messages after receiving the audio and video segments to avoid the situation that the server slows down due to not receiving the ACK message.
  • the server will actively reduce the playback bit rate. This method greatly reduces the situation that the audio and video playback process is stuck.
  • multiple ACK messages are sent to the server, including:
  • the preset condition includes that the signal strength is less than the preset strength threshold, and/or the network delay is greater than the preset delay threshold.
  • the terminal device when the current signal strength of the access network is less than the preset strength threshold, and/or the network delay is greater than the preset delay threshold, the terminal device sends multiple ACK messages to the server, that is, the terminal device can It can flexibly control the sending mode of the ACK message according to the network quality situation of the terminal device, and only send multiple ACK messages when the network quality of the current access network of the terminal device is poor, which further improves the intelligence and flexibility of the terminal device.
  • the number of sent ACK messages is negatively correlated with the signal strength of the access network.
  • the number of ACK messages sent is negatively correlated with the signal strength of the access network, that is, the higher the signal strength of the access network, the less redundancy of the ACK message.
  • the embodiments of the present application provide a method for processing audio and video, which is used in a server, and the method includes:
  • the network quality information includes the network bandwidth of the access network, and according to the audio and video playback request, the audio and video data of the target audio and video is returned to the terminal device, including:
  • the network bandwidth of the access network and the network bandwidth of the server determine the size of the communication window and the playback code rate
  • the audio and video data of the target audio and video are returned to the terminal device.
  • the server determines the size of the communication window and the playback code rate according to the network quality information actively reported by the terminal device, and returns the audio and video data of the target audio and video according to the size of the communication window and the playback code rate, so as to ensure the first quality of the target audio and video as much as possible.
  • Each audio and video segment is sent to the terminal device as soon as possible, skipping the slow start process of the TCP protocol in the related art, and realizing the fast start of audio and video playback.
  • the audio and video data includes at least one audio and video segment, and the method further includes:
  • the method further includes:
  • the duplicate ACK messages are removed.
  • the server in the case of receiving multiple ACK messages corresponding to the same audio and video segment, the server removes the duplicate ACK messages, thereby realizing the de-redundancy processing of the ACK messages.
  • embodiments of the present application provide an apparatus for processing audio and video, the apparatus includes at least one unit, and the at least one unit is configured to implement the first aspect or any one of the possible implementation manners of the first aspect. provided method.
  • an embodiment of the present application provides an audio and video processing device, the device includes at least one unit, and the at least one unit is configured to implement the second aspect or any one of the possible implementation manners of the second aspect. provided method.
  • embodiments of the present application provide an apparatus for processing audio and video, the apparatus comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned first step when executing the instructions
  • a processor comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned first step when executing the instructions
  • a method provided by any one of the possible implementations of the aspect or the first aspect.
  • embodiments of the present application provide an apparatus for processing audio and video, the apparatus comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned second when executing the instructions A method provided by any one of the possible implementations of the aspect or the second aspect.
  • embodiments of the present application provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic device When running, the processor in the electronic device executes the method provided by the first aspect or any one of the possible implementation manners of the first aspect.
  • embodiments of the present application provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic device
  • the processor in the electronic device executes the method provided by the second aspect or any one of the possible implementation manners of the second aspect.
  • embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the first aspect or the first aspect is implemented.
  • embodiments of the present application provide a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the second aspect or the second aspect above.
  • FIG. 1 shows a schematic diagram of the positions of different influencing factors in the video playback process in the related art.
  • FIG. 2 shows a schematic flow chart of an execution flow of RTMP in the related art.
  • FIG. 3 shows a schematic structural diagram of an audio and video processing system provided by an exemplary embodiment of the present application.
  • FIG. 4 shows a flowchart of a method for processing audio and video provided by an exemplary embodiment of the present application.
  • FIG. 5 shows a flowchart of a method for processing audio and video in the related art.
  • FIG. 6 shows a flowchart of an audio and video processing method provided by another exemplary embodiment of the present application.
  • FIG. 7 shows a flowchart of a method for processing audio and video provided by another exemplary embodiment of the present application.
  • FIG. 8 shows a block diagram of an apparatus for processing audio and video provided by an exemplary embodiment of the present application.
  • FIG. 9 shows a block diagram of an audio and video processing apparatus provided by an exemplary embodiment of the present application.
  • FIG. 10 shows a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • FIG. 11 shows a schematic structural diagram of a server provided by an embodiment of the present application.
  • the terminal device starts to download segments after sending a play request, and the terminal device starts to play the video after the amount of downloaded data reaches the pre-buffering threshold, which affects the first buffering time.
  • the fragmented download process a video is divided into multiple fragments of different qualities, which are distributed and scheduled on different servers, and downloaded and played in units of fragments. This process affects the occurrence of stuck events. If the terminal device adopts the strategy of "downloading the currently playing segment as soon as possible", making full use of the currently available bandwidth, and downloading while playing, this process affects the traffic model.
  • the terminal device adopts the "intermittent download” strategy, that is, it will not download the unplayed segments all the time. If the remaining playback time is greater than the pause buffer threshold, the download will be paused, and if the remaining playback time is less than the pause buffer threshold, the download will be re-downloaded. Process affects the occurrence of Caton events.
  • RTMP Content Delivery Network
  • HTTP Hypertext Transfer Protocol
  • Adaptive streaming media Dynamic Adaptive Streaming over HTTP, DASH
  • RTMP is a transmission protocol for audio and video playback in a live broadcast scenario.
  • the DASH protocol is the main protocol for audio and video playback in on-demand scenarios.
  • the server stores codes of various resolutions and dynamically selects them according to network conditions.
  • the execution flow of RTMP includes but is not limited to the following steps.
  • Step 201 the terminal device sends a connection request to the server;
  • Step 202 the server determines the size of the communication window for sending content messages according to its own bandwidth, and sends the size of the communication window to the terminal device;
  • Step 203 the server sends the set bandwidth information to the terminal device
  • Step 204 the terminal device determines the negotiated communication window size according to the network bandwidth of the current access network and the bandwidth information sent by the server, and sends the negotiated communication window size to the server, thereby negotiating and completing the communication window size of the transmission data between the two;
  • Step 205 the terminal device initiates an audio and video playback request to the server;
  • Step 206 the server sends a confirmation message of the size of the content block to the terminal device;
  • Step 207 the server sends a playback operation execution success message to the terminal device;
  • Step 208 The server sends audio and video data to the terminal device.
  • the server after the terminal device initiates a connection request, the server usually needs at least 2 to 3 RTTs for network bandwidth detection.
  • the embodiments of the present application provide an audio and video processing method, device, and storage medium.
  • a terminal device sends an audio and video playback request to a server, it actively reports a network for indicating the network quality of the current access network of the terminal device.
  • the quality information avoids the situation of multiple interactions between the server and the terminal device for network bandwidth detection in the related art, thereby shortening the playback delay of audio and video, and ensuring the playback effect of audio and video.
  • FIG. 3 shows a schematic structural diagram of an audio and video processing system provided by an exemplary embodiment of the present application.
  • the system includes a terminal device 120 and a server 140 .
  • the terminal device 120 runs an audio and video client.
  • the audio and video client is a software application for playing audio and video. Users can play audio and video through the audio and video client.
  • the terminal device 120 is configured to send an audio and video playback request to the server 140 through the audio and video client, receive the audio and video data returned by the server 140, and complete the audio and video playback in the audio and video client.
  • the terminal device 120 is a mobile phone, a car terminal, a tablet computer, an e-book reader, a moving picture expert compression standard audio layer 3 (Moving Picture Experts Group Audio Layer III, MP3) player, and a moving picture expert compression standard audio layer 4 ( Moving Picture Experts Group Audio Layer IV, MP4) Players, Notebooks, Laptops and Desktops, etc.
  • This embodiment of the present application does not limit the type of the terminal device 120 .
  • a communication connection is established between the terminal device 120 and the server 140, and the communication connection may be a wired network or a wireless network.
  • the server 140 is also called a media content server, and is configured to return audio and video data after receiving the audio and video playback request sent by the terminal device 120 .
  • the server 140 is a CDN server.
  • the server 140 is an audio server or a video server.
  • the terminal device 120 is used to receive a playback instruction, and the playback instruction is used to instruct to start playing the target audio and video;
  • the audio and video playback request is used to instruct the server 140 to return the audio and video data of the target audio and video; download and play the audio and video data of the target audio and video.
  • FIG. 4 shows a flowchart of an audio and video processing method provided by an exemplary embodiment of the present application. This embodiment is illustrated by using the method in the terminal device shown in FIG. 3 . The method includes but is not limited to the following steps.
  • step 401 the terminal device receives a play instruction, where the play instruction is used to instruct to start playing the target audio and video.
  • the terminal device displays a user interface of the audio and video client, and the user interface includes playback controls of the target audio and video.
  • the terminal device receives a user operation signal acting on the playback control, that is, receives a playback instruction, step 402 is performed.
  • the user interface of the audio and video client displays a plurality of audio and video playback controls corresponding to each other.
  • the target audio and video is any one of multiple audio and video.
  • Step 402 The terminal device sends an audio and video playback request carrying network quality information to the server according to the playback instruction.
  • the network quality information is used to indicate the current network quality of the terminal device accessing the network, and the audio and video playback request is used to instruct the server to return the audio and video data of the target audio and video.
  • the terminal device acquires network quality information of the current access network of the terminal device.
  • the terminal device sends an audio and video playback request to the server, where the audio and video playback request carries network quality information. That is, when the terminal device sends an audio and video playback request to the server, it actively reports network quality information according to system information and historical information.
  • the network quality information is used to indicate the network quality of the current access network of the terminal device.
  • the network quality information may include the network bandwidth of the current access network of the terminal device.
  • the network quality information may also include the current signal strength and/or network delay of the access network.
  • the audio and video playback request carries the identification of the target audio and video
  • the audio and video playback request is used to instruct the server to return audio and video data corresponding to the identification of the target audio and video.
  • Step 403 The server receives an audio and video playback request that carries network quality information and is sent by the terminal device.
  • the network quality information is used to indicate the network quality of the current access network of the terminal device.
  • the server After receiving the audio and video playback request sent by the terminal device, the server obtains the network quality information carried in the audio and video playback request.
  • Step 404 the server returns the audio and video data of the target audio and video to the terminal device according to the audio and video playback request.
  • the server determines the size of the communication window and the playback code rate according to the network quality information; and returns the audio and video data of the target audio and video to the terminal device according to the size of the communication window and the playback code rate.
  • the audio and video data includes at least one audio and video segment
  • the server sequentially returns at least one audio and video segment of the target audio and video to the terminal device.
  • Step 405 the terminal device downloads and plays the audio and video data of the target audio and video.
  • the terminal device receives at least one audio and video segment of the target audio and video sequentially returned by the server. For each audio and video fragment, the terminal device downloads and plays the audio and video fragment after receiving the audio and video fragment returned by the server.
  • an audio and video playback request carrying network quality information is sent to the server, and the audio and video playback request is used to instruct the server to return the target audio and video audio.
  • Video data the terminal device downloads and plays the audio and video data of the target audio and video returned by the server; that is, when the terminal device sends an audio and video playback request to the server, it actively reports the network used to indicate the network quality of the current access network of the terminal device.
  • the quality information avoids the situation of multiple interactions between the server and the terminal device for network bandwidth detection in the related art, thereby shortening the playback delay of audio and video, and ensuring the playback effect of audio and video.
  • the control end that sends data is on the server side, but the network bottleneck is often on the terminal device side, which leads to the following two problems:
  • the feedback of the network quality on the terminal device side is slow.
  • the server sends an initial message to the terminal device after receiving the connection request sent by the terminal device; in step 502, the terminal device feeds back an ACK message; in step 503, the server sends a second message to the terminal device, the second message
  • the packet size of the message is larger than that of the initial packet; in step 504, the terminal device feeds back an ACK message.
  • the server when the server detects the network bandwidth, it usually takes at least 2 to 3 RTTs to detect the network bandwidth.
  • the link between the end device and the server is not symmetrical.
  • the server after detecting the network bandwidth of the current access network of the terminal device, the server adjusts the size of the communication window and the playing bit rate of audio and video according to the network bandwidth.
  • the server downloads the pronunciation video segment. After the terminal device actually receives the audio and video segments, it will return an ACK message to the server.
  • Step 507 If the server does not receive the ACK message sent by the terminal device, it considers that the network is damaged, immediately reduces the audio and video playback bit rate, and sends subsequent audio and video segments according to the reduced playback bit rate. If the server does not receive more than the preset number of ACK messages, it will stop sending data. As a result, the image quality of the audio and video playback interface is degraded or the audio and video playback process freezes.
  • the embodiment of the present application provides an audio and video processing method, including but not limited to It is limited to the following two stages, as shown in Figure 6: the first stage is the initial link establishment stage, step 601, the terminal device sends a connection request to the server through the audio and video client; step 602, the server returns an ACK message to the terminal device. Step 603, the terminal device sends an audio and video playback request carrying network quality information to the server through the audio and video client, and the network quality information includes the network bandwidth of the terminal device's current access network; Step 604, the server according to the audio and video playback request.
  • the second stage is the data transmission stage.
  • the server sequentially returns at least one audio and video segment of the target audio and video to the terminal device according to the determined communication window size and playback bit rate.
  • the terminal device after receiving the audio and video segments, the terminal device sends multiple ACK messages to the server to ensure that they are received by the server, so as to prevent the server from reducing the playback bit rate due to the failure to receive the ACK message in the related art, so that the picture quality is degraded, Even reduce the size of the communication window, resulting in a stuck situation.
  • Step 607 In the case of receiving multiple ACK messages corresponding to the same audio and video segment, the server removes the duplicate ACK messages.
  • FIG. 7 shows a flowchart of an audio and video processing method provided by another exemplary embodiment of the present application. This embodiment is illustrated by using the method in the terminal device shown in FIG. 3 . The method includes but is not limited to the following steps.
  • Step 701 the terminal device acquires the network quality information of the current access network.
  • the terminal device Before receiving the play instruction, the terminal device extracts and acquires the network quality information of the current access network.
  • the terminal device acquires the network quality information in real time or every preset time period or when a preset trigger signal is received.
  • the preset time interval is set by default, or is custom set.
  • the preset trigger signal is a user operation signal acting on the user interface of the audio and video client.
  • the preset trigger signal can be a user interface switching signal, and can also be a user interface refresh signal.
  • the preset trigger signal includes any one or a combination of a click operation signal, a sliding operation signal, a pressing operation signal, and a long-pressing operation signal. This embodiment of the present application does not limit this.
  • Step 702 The terminal device sends an audio and video playback request carrying network quality information to the server according to the received playback instruction.
  • the terminal device After receiving the playback instruction, the terminal device sends an audio and video playback request carrying network quality information to the server.
  • Step 703 the server determines the size of the communication window and the playing bit rate according to the network bandwidth of the access network and the network bandwidth of the server.
  • the server receives the audio and video playback request sent by the terminal device, and obtains the network quality information carried in the audio and video playback request.
  • the network quality information includes the network bandwidth of the current access network of the terminal device.
  • the network quality information further includes the current signal strength and/or network delay of the access network.
  • the server determines the target network bandwidth according to the network bandwidth of the access network and the network bandwidth of the server; according to the preset correspondence, determines the communication window size and playback code rate corresponding to the target network bandwidth, and the preset correspondence includes: Correspondence between network bandwidth, communication window size and playback bit rate.
  • the embodiments of the present application do not limit the manner in which the size of the communication window and the playing bit rate are determined according to the network bandwidth.
  • the server determines the initial communication window size and playback bit rate according to the network bandwidth of the access network and the network bandwidth of the server, so as to ensure that the first audio and video segment of the target audio and video is sent to the terminal device as soon as possible.
  • Step 704 the server returns the audio and video segments of the target audio and video to the terminal device according to the size of the communication window and the playing bit rate.
  • the server sequentially returns at least one audio and video segment of the target audio and video to the terminal device according to the determined communication window size and playback bit rate.
  • the server returns the first audio and video segment of the target audio and video to the terminal device according to the determined initial communication window size and playback code rate. After receiving the ACK message corresponding to the audio and video fragment returned by the terminal device, the server continues to return the second audio and video fragment of the target audio and video to the terminal device. And so on, and will not repeat them.
  • Step 705 After receiving the audio and video segments, the terminal device sends multiple ACK messages to the server.
  • the ACK message is used to indicate that the terminal device has successfully received the audio and video segments.
  • the terminal device sends multiple ACK messages to the server after receiving the audio and video segments, where the audio and video segments are at least one of the target audio and video segments. anyone.
  • the terminal device after receiving the audio and video segments, the terminal device returns one or more ACK messages according to the collected current signal strength and/or network delay of the access network.
  • the multiple ACK messages are at least two ACK messages.
  • the terminal device when the current signal strength and/or network delay of the access network meet a preset condition, the terminal device sends multiple ACK messages to the server; wherein the preset condition includes that the signal strength is less than a preset strength threshold, and/ Or the network delay is greater than the preset delay threshold.
  • the signal strength and/or network delay of the access network is used to indicate the network quality of the access network.
  • the signal strength of the access network is positively correlated with the network quality, that is, the stronger the signal strength of the access network, the better the network quality of the access network.
  • the network delay of the access network is negatively correlated with the network quality, that is, the greater the network delay of the access network, the worse the network quality of the access network.
  • the network delay of the access network is also called RTT.
  • the preset intensity threshold or the preset delay threshold is a default setting or a custom setting. This embodiment of the present application does not limit this.
  • the terminal device determines whether the current signal strength and/or network delay of the access network meets the preset conditions, if the current signal strength and/or network delay of the access network Or if the network delay satisfies the preset condition, the number of ACK messages to be sent and the sending interval are determined, and multiple ACK messages are sent in sequence according to the sending interval; if the preset condition is not met, an ACK message is sent to the server.
  • the number of ACK messages sent is negatively correlated with the signal strength of the access network. That is, the weaker the signal strength of the current access network of the terminal device, the greater the number of ACK messages sent.
  • the number of sent ACK messages is set to a minimum value and a maximum value. For example, the minimum value is 1 and the maximum value is 3. This embodiment of the present application does not limit this.
  • the sending interval is a time interval between two ACK messages sent in sequence, and multiple sending intervals may be the same or different. This embodiment of the present application does not limit this.
  • the terminal device after receiving the audio and video segments, the terminal device sends an ACK message to the server, and judges whether the current signal strength and/or network delay of the access network meets the preset conditions. If the current signal strength and/or network delay of the incoming network meets the preset condition, the terminal device determines the number of ACK messages to be sent. The terminal device detects the current signal strength of the access network in real time, and sends an ACK message again when the signal strength is greater than a preset strength threshold.
  • the embodiment of the present application does not limit the sending manner of multiple ACK messages.
  • Step 706 The server removes duplicate ACK messages in the case of receiving multiple ACK messages corresponding to the same audio and video segment.
  • the server In order to avoid misoperation, when the server receives multiple ACK messages corresponding to the same audio and video segment, it removes the duplicate ACK messages and re-estimates the network delay. After receiving the ACK message corresponding to the audio and video segments, the server keeps the size of the communication window and the playback bit rate unchanged, and continues to perform the step of returning the audio and video segments of the target audio and video to the terminal device according to the size of the communication window and the playback bit rate .
  • the mobile phone accesses the home network A through wireless Internet access (English: Wi-Fi), and starts the K video client.
  • the home network A detects that the bandwidth of the current home network A is 50 Mbps, and when initiating a video playback request corresponding to the video "XX trouble", it notifies the video server that the bandwidth of the home network A currently connected by the mobile phone is 50 Mbps.
  • the video server selects the best video playback bit rate of 1080p according to its own bandwidth and the bandwidth of the current home network A of the mobile phone, and plays it at 15Mbps.
  • the first segment is assumed to be 2s long, about 30Mb in size, and the network delay RTT from the video server to the mobile phone is about 100ms, then the mobile phone receives the click operation of the "Play” button until the video starts to play. (i.e. the start-up delay) is about 100ms, that is, the video "XX trouble” can be played after 100ms.
  • the start-up delay is about 100ms, that is, the video "XX trouble” can be played after 100ms.
  • TCP Transmission Control Protocol
  • additional 3-5 RTTs are required to The confirmation of the normal content block size is completed, and the start-up delay is at least 500ms.
  • the video server During the video playback process, if the user walks around with the mobile phone to a location with weak signal, considering the impact of obstacles on Wi-Fi, data packets may be lost, causing the video server to push video fragments to the terminal device. , the ACK message from the terminal device has not been received, but if the user continues to move to a position with a strong signal, at this time, the video clip has not been played, and the terminal device detects that the signal strength is higher than the preset strength threshold and sends it to the video server again.
  • the ACK message corresponding to the video clip after the video server obtains the ACK message, normally pushes the subsequent video clips of the 1080p playback bit rate.
  • the video server directly reduces the playback bit rate of the video after not receiving the ACK message, resulting in a decrease in image quality, and due to the change in the playback bit rate, time delay may occur, and video playback may become stuck.
  • the audio and video processing method provided by the embodiment of the present application optimizes the key indicators in the audio and video playback scene:
  • Startup delay that is, the time from receiving the playback instruction to the start of audio and video playback.
  • This indicator is related to the current network bandwidth, network delay and audio and video playback bit rate of the terminal device's current access network. For example, if the video to be played is 1080p video, the network bandwidth is about 15Mbps, the first video segment of the video is 2s, and the size of the video segment is 30Mb; the user's home network bandwidth is 50Mb, and the delay is 100ms.
  • the terminal device actively reports that the network bandwidth of the access network is 50Mb, the server transmits through 30Mb, and the terminal device can download and play the first audio and video segment of the audio and video in one RTT.
  • the start-up delay will be reduced from 500ms to 100ms.
  • the network packet loss rate that is, the ratio of the number of lost packets to the transmitted packets.
  • Packet loss will affect the continuity of audio and video playback. If packet loss occurs on the network, the audio and video playback process may freeze. For example, the network packet loss rate is 5%. The packet loss rate of the terminal device responding with a single ACK message is 5%. After the packet loss, the server reduces the playback bit rate of the audio and video, which affects the transmission throughput of the audio and video. That is, the probability of a freeze during the audio and video playback process is about 5%. %.
  • the terminal device responds with multiple ACK messages.
  • the packet loss rate of using two ACK messages to respond is 0.25%, that is, only a 0.25% probability will cause the server to reduce the playback code of audio and video.
  • rate that is, the probability of stuttering during audio and video playback is only 0.25%.
  • the terminal device actively reports network quality information: the terminal device sends an audio and video playback request to the server, and the audio and video playback request carries the current information of the terminal device.
  • the network quality information of the access network including the network bandwidth, avoids the situation in the related art that the network quality information needs to be acquired by the server through multiple RTT detections.
  • the quick start of audio and video playback is realized: the server determines the size of the communication window and the playback code rate according to the network quality information actively reported by the terminal device, and returns the audio and video data of the target audio and video according to the size of the communication window and the playback code rate, Try to ensure that the first audio and video segment of the target audio and video is sent to the terminal device as soon as possible, skipping the slow start process of the TCP protocol in the related art.
  • the access network congestion processing is implemented: in order to prevent the loss of ACK messages, the terminal device sends multiple ACK messages after receiving the audio and video segments to avoid the server from downgrading due to not receiving the ACK message.
  • the server In the case of high speed, the number of ACK messages sent is negatively correlated with the signal strength of the access network, that is, the higher the signal strength of the access network, the less redundancy of ACK messages; only when multiple ACK messages are lost at the same time In the case of packets, the server will actively reduce the playback bit rate. This method greatly reduces the situation that the audio and video playback process is stuck; on the other hand, the de-redundancy processing of ACK messages is realized: the server receives In the case of multiple ACK messages corresponding to the same audio and video segment, remove the duplicate ACK messages and re-estimate the RTT.
  • the server does not need to reduce the audio and video playback bit rate, and can still return the next audio and video fragment to the terminal device according to the current communication window size and playback bit rate. If neither is received, the server will reduce the playback bit rate of the audio and video, and send the subsequent audio and video segments according to the reduced playback bit rate.
  • FIG. 8 shows a block diagram of an apparatus for processing audio and video provided by an exemplary embodiment of the present application.
  • the audio and video processing apparatus can be implemented by software, hardware or a combination of the two to become all or a part of the terminal device shown in FIG. 3 .
  • the audio and video processing apparatus may include: a receiving unit 810 , a sending unit 820 and a processing unit 830 .
  • the receiving unit 810 is used to receive a play instruction, and the play instruction is used to instruct to start playing the target audio and video;
  • the sending unit 820 is configured to send an audio and video playback request carrying network quality information to the server according to the playback instruction, where the network quality information is used to indicate the network quality of the current access network of the terminal device, and the audio and video playback request is used to indicate the server. Return the audio and video data of the target audio and video;
  • the processing unit 830 downloads and plays the audio and video data of the target audio and video.
  • the network quality information includes the network bandwidth of the current access network of the terminal device.
  • the audio and video data includes at least one audio and video segment
  • the device further includes:
  • the sending unit 820 is further configured to send multiple ACK messages to the server after receiving the audio and video segments, where the ACK messages are used to indicate that the terminal device has successfully received the audio and video segments.
  • the sending unit 820 is further configured to, after receiving the audio and video segments, when the current signal strength and/or network delay of the access network meets a preset condition, send the multiple ACK messages;
  • the preset condition includes that the signal strength is less than the preset strength threshold, and/or the network delay is greater than the preset delay threshold.
  • the number of sent ACK messages is negatively correlated with the signal strength of the access network.
  • FIG. 9 shows a block diagram of an apparatus for processing audio and video provided by an exemplary embodiment of the present application.
  • the audio and video processing apparatus can be implemented by software, hardware or a combination of the two to become all or a part of the server shown in FIG. 3 .
  • the audio and video processing apparatus may include: a receiving unit 910 and a sending unit 920 .
  • a receiving unit 910 configured to receive an audio and video playback request that carries network quality information sent by the terminal device, where the network quality information is used to indicate the current network quality of the terminal device's access network;
  • the sending unit 920 is configured to return the audio and video data of the target audio and video to the terminal device according to the audio and video playback request.
  • the network quality information includes the network bandwidth of the access network
  • the sending unit 920 is further configured to:
  • the network bandwidth of the access network and the network bandwidth of the server determine the size of the communication window and the playback code rate
  • the audio and video data of the target audio and video are returned to the terminal device.
  • the audio and video data includes at least one audio and video segment
  • the device further includes:
  • the receiving unit 910 is further configured to receive multiple ACK messages corresponding to the audio and video segments sent by the terminal device, where the ACK messages are used to indicate that the terminal device has successfully received the audio and video segments.
  • the apparatus further includes: a processing unit;
  • the processing unit is configured to remove duplicate ACK messages in the case of receiving multiple ACK messages corresponding to the same audio and video segment.
  • FIG. 10 shows a schematic structural diagram of a terminal device provided by an embodiment of the present application.
  • the terminal device includes a central processing unit (Central Processing Unit, CPU) 1010, a memory 1020, and a network interface 1030.
  • CPU Central Processing Unit
  • the central processing unit 1010 includes one or more processing cores.
  • the central processing unit 1010 is used to execute various functional applications of the terminal device and to perform data processing.
  • Terminal devices typically include multiple network interfaces 1030 .
  • the memory 1020 is connected to the central processing unit 1010 through a bus.
  • the memory 1020 is used to store instructions, and the processor 1010 implements the above-mentioned audio and video processing method executed by the terminal device by executing the instructions stored in the memory 1020.
  • the memory 1020 may store an operating system 1021 and an application program module 1022 required for at least one function.
  • the operating system 1021 includes at least one of a LINUX operating system, a Unix operating system, and a Windows operating system.
  • the application module 1022 includes a receiving unit, a sending unit, a processing unit, and other units for implementing the above-mentioned audio and video processing methods, and the like.
  • a receiving unit for receiving a playback instruction, and the playback instruction is used for instructing to start playing the target audio and video;
  • the sending unit is used to send an audio and video playback request carrying network quality information to the server according to the playback instruction, where the network quality information is used to indicate the network quality of the current access network of the terminal device, and the audio and video playback request is used to instruct the server to return The audio and video data of the target audio and video;
  • the processing unit downloads and plays the audio and video data of the target audio and video.
  • memory 1020 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read only memory
  • EPROM Erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • magnetic or Optical Disk any type of volatile or non-volatile storage device or combination thereof
  • FIG. 11 shows a schematic structural diagram of a server provided by an embodiment of the present application.
  • the server includes a CPU 1110 , a memory 1120 , and a network interface 1130 .
  • the central processing unit 1110 includes one or more processing cores.
  • the central processing unit 1110 is used to execute various functional applications of the server and to perform data processing.
  • the server typically includes multiple network interfaces 1130 .
  • the memory 1120 is connected to the central processing unit 1110 through a bus.
  • the memory 1120 is used for storing instructions, and the processor 1110 implements the above-mentioned audio and video processing method executed by the server by executing the instructions stored in the memory 1120 .
  • the memory 1120 may store an operating system 1121 and an application module 1122 required for at least one function.
  • the operating system 1121 includes at least one of a LINUX operating system, a Unix operating system, and a Windows operating system.
  • the application module 1122 includes a receiving unit, a sending unit, and other units for implementing the above-mentioned audio and video processing methods, and the like.
  • a receiving unit configured to receive an audio and video playback request that carries network quality information sent by the terminal device, and the network quality information is used to indicate the current network quality of the terminal device's access network;
  • the sending unit is configured to return the audio and video data of the target audio and video to the terminal device according to the audio and video playback request.
  • memory 1120 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read only memory
  • EPROM Erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • magnetic or Optical Disk any type of volatile or non-volatile storage device or combination thereof
  • An embodiment of the present application provides a terminal device, the terminal device includes: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned method executed on the terminal device side when executing the instructions.
  • An embodiment of the present application provides a server, which includes: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above server-side execution method when executing the instructions.
  • Embodiments of the present application provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are executed in a processor of an electronic device , the processor in the electronic device executes the above method.
  • Embodiments of the present application provide a non-volatile computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.
  • a computer-readable storage medium may be a tangible device that retains and stores instructions for use by the instruction execution device.
  • examples of computer-readable storage media include, but are not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), static random access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing .
  • RAM random access memory
  • ROM read only memory
  • EPROM erasable programmable read-only memory
  • EPROM Errically Programmable Read-Only-Memory
  • SRAM static random access memory
  • portable compact disk read-only memory Compact Disc Read-Only Memory
  • CD - ROM Compact Disc Read-Only Memory
  • DVD Digital Video Disc
  • memory sticks floppy disks
  • Computer readable program instructions or code described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present application may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, may be connected to an external computer (eg, use an internet service provider to connect via the internet).
  • electronic circuits such as programmable logic circuits, Field-Programmable Gate Arrays (FPGA), or Programmable Logic Arrays (Programmable Logic Arrays), are personalized by utilizing state information of computer-readable program instructions.
  • Logic Array, PLA the electronic circuit can execute computer readable program instructions to implement various aspects of the present application.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.
  • Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented in hardware (eg, circuits or ASICs (Application) that perform the corresponding functions or actions. Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented by a combination of hardware and software, such as firmware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请涉及本申请涉及多媒体技术领域，尤其涉及一种音视频的处理方法、装置及存储介质。该方法用于终端设备中，该方法包括：接收播放指令，播放指令用于指示开始播放目标音视频；根据播放指令，向服务器发送携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况，音视频播放请求用于指示服务器返回目标音视频的音视频数据；下载并播放目标音视频的音视频数据。本申请实施例通过在终端设备向服务器发送音视频播放请求时，主动上报终端设备当前的接入网络的网络质量信息，避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况，从而缩短了音视频的起播时延，保证了音视频的播放效果。

Description

音视频的处理方法、装置及存储介质
本申请要求于2020年11月18日提交中国专利局、申请号为202011292718.4、申请名称为“音视频的处理方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及多媒体技术领域，尤其涉及一种音视频的处理方法、装置及存储介质。
背景技术
目前，音视频播放一般采用基于音视频分片的下载模式。例如，通过互联网向终端设备提供音视频的下载方式中，服务器将同一个音视频的不同码率的版本切分为预设长度的分片，对每个分片进行封装。
在终端设备需要对音视频进行播放时，向服务器发送连接请求，服务器在接收到连接请求后，通过与终端设备的多次交互进行带宽探测，从而确定合适的网络通信初始窗口大小。服务器在接收到终端设备发送的音视频播放请求后，根据确定出的网络通信初始窗口大小传输音视频的初始分片。
但是在上述方法中，服务器在客户端发送连接请求后，需要经过多个网络通信往返时延(Round trip time，RTT)进行带宽探测，导致音视频的起播时延较长，音视频播放效果不佳。
发明内容
有鉴于此，提出了一种音视频的处理方法、装置及存储介质，在终端设备向服务器发送音视频播放请求时，主动上报用于指示终端设备当前的接入网络的网络质量情况的网络质量信息，避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况，从而缩短了音视频的起播时延，保证了音视频的播放效果。
第一方面，本申请的实施例提供了一种音视频的处理方法，用于终端设备中，该方法包括：
接收播放指令，播放指令用于指示开始播放目标音视频；
根据播放指令，向服务器发送携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况，音视频播放请求用于指示服务器返回目标音视频的音视频数据；
下载并播放目标音视频的音视频数据。
在该实现方式中，终端设备接收到目标音视频的播放指令后，向服务器发送携带有网络质量信息的音视频播放请求，音视频播放请求用于指示服务器返回目标音视频的音视频数据；终端设备下载并播放服务器返回的目标音视频的音视频数据；即在终端设备向服务器发送音视频播放请求时，主动上报终端设备当前的接入网络的网络质量信息，避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况， 从而缩短了音视频的起播时延，保证了音视频的播放效果。
在一种可能的实现方式中，网络质量信息包括终端设备当前的接入网络的网络带宽。
在该实现方式中，终端设备主动上报终端设备当前的接入网络的网络带宽，避免了相关技术中网络带宽需要服务器经过多个RTT探测才能获取的情况。
在另一种可能的实现方式中，音视频数据包括至少一个音视频分片，该方法还包括：
在接收到音视频分片后，向服务器发送多个确认应答(Acknowledge，ACK)消息，ACK消息用于指示终端设备已成功接收到音视频分片。
在该实现方式中，为了防止ACK消息丢包，终端设备在收到音视频分片后，通过发送多个ACK消息，来避免服务器由于未收到ACK消息而进行降速的情况，只有在多个ACK消息同时丢包的情况下，服务器才会主动下降播放码率，这种方法极大地降低了音视频播放过程发生卡顿的情况。
在另一种可能的实现方式中，在接收到音视频分片后，向服务器发送多个ACK消息，包括：
在接收到音视频分片后，当接入网络当前的信号强度和/或网络时延满足预设条件时，向服务器发送多个ACK消息；
其中，预设条件包括信号强度小于预设强度阈值，和/或网络时延大于预设时延阈值。
在该实现方式中，当接入网络当前的信号强度小于预设强度阈值，和/或网络时延大于预设时延阈值时，终端设备向服务器发送多个ACK消息，即终端设备能够根据当前的网络质量情况灵活地控制ACK消息的发送方式，在终端设备当前的接入网络的网络质量较差的情况下才会发送多个ACK消息，进一步提高了终端设备的智能性和灵活性。
在另一种可能的实现方式中，ACK消息的发送个数与接入网络的信号强度呈负相关关系。
在该实现方式中，ACK消息的发送个数与接入网络的信号强度呈负相关关系，即接入网络的信号强度越高，ACK消息的冗余量越少。
第二方面，本申请的实施例提供了一种音视频的处理方法，用于服务器中，该方法包括：
接收终端设备发送的携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况；
根据音视频播放请求，向终端设备返回目标音视频的音视频数据。
在一种可能的实现方式中，网络质量信息包括接入网络的网络带宽，根据音视频播放请求，向终端设备返回目标音视频的音视频数据，包括：
根据接入网络的网络带宽和服务器的网络带宽，确定通信窗口大小和播放码率；
根据通信窗口大小和播放码率，向终端设备返回目标音视频的音视频数据。
在该实现方式中，服务器根据终端设备主动上报的网络质量信息确定通信窗口大小和播放码率，根据通信窗口大小和播放码率返回目标音视频的音视频数据，尽量保 障目标音视频的第一个音视频分片尽快发送至终端设备，跳过了相关技术中的TCP协议的慢启动过程，实现了音视频播放的快速启动。
在另一种可能的实现方式中，音视频数据包括至少一个音视频分片，该方法还包括：
接收终端设备发送的音视频分片对应的多个ACK消息，ACK消息用于指示终端设备已成功接收到音视频分片。
在另一种可能的实现方式中，该方法还包括：
在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息。
在该实现方式中，服务器在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息，实现了ACK消息的去冗余处理。
第三方面，本申请的实施例提供了一种音视频的处理装置，该装置包括至少一个单元，至少一个单元用于实现上述第一方面或第一方面中的任意一种可能的实现方式所提供的方法。
第四方面，本申请的实施例提供了一种音视频的处理装置，该装置包括至少一个单元，至少一个单元用于实现上述第二方面或第二方面中的任意一种可能的实现方式所提供的方法。
第五方面，本申请的实施例提供了一种音视频的处理装置，装置包括：处理器；用于存储处理器可执行指令的存储器；其中，处理器被配置为执行指令时实现上述第一方面或第一方面中的任意一种可能的实现方式所提供的方法。
第六方面，本申请的实施例提供了一种音视频的处理装置，装置包括：处理器；用于存储处理器可执行指令的存储器；其中，处理器被配置为执行指令时实现上述第二方面或第二方面中的任意一种可能的实现方式所提供的方法。
第七方面，本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当计算机可读代码在电子设备中运行时，电子设备中的处理器执行上述第一方面或者第一方面中的任意一种可能的实现方式所提供的方法。
第八方面，本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当计算机可读代码在电子设备中运行时，电子设备中的处理器执行上述第二方面或者第二方面中的任意一种可能的实现方式所提供的方法。
第九方面，本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令被处理器执行时实现上述第一方面或者第一方面中的任意一种可能的实现方式所提供的方法。
第十方面，本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令被处理器执行时实现上述第二方面或第二方面中的任意一种可能的实现方式所提供的方法。
附图说明
包含在说明书中并且构成说明书的一部分的附图与说明书一起示出了本申请的示例性实施例、特征和方面，并且用于解释本申请的原理。
图1示出相关技术中不同的影响因素在视频播放过程中的位置示意图。
图2示出相关技术中RTMP的执行流程的流程示意图。
图3示出了本申请一个示例性实施例提供的音视频的处理***的结构示意图。
图4示出了本申请一个示例性实施例提供的音视频的处理方法的流程图。
图5示出相关技术中音视频的处理方法的流程图。
图6示出了本申请另一个示例性实施例提供的音视频的处理方法的流程图。
图7示出了本申请另一个示例性实施例提供的音视频的处理方法的流程图。
图8示出了本申请一个示例性实施例提供的音视频的处理装置的框图。
图9示出了本申请一个示例性实施例提供的音视频的处理装置的框图。
图10示出了本申请一个实施例提供的终端设备的结构示意图。
图11示出了本申请一个实施例提供的服务器的结构示意图。
具体实施方式
以下将参考附图详细说明本申请的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本申请，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。
在音视频传输技术中，如何反馈网络质量情况，并根据网络质量情况对音视频的播放码率进行调整，从而获取更佳的流畅度和更短的起播时延，是音视频传输技术中的关键。以视频传输为例，在视频播放过程中，影响播放效果的多种影响因素包括但不限于：请求的建链时间、起始分片的返回时间、分片大小及分片下载策略等。不同的影响因素在视频播放过程中的位置示意图如图1所示。其中，在预先缓冲过中，终端设备发送播放请求后开始下载分片，下载数据量达到预先缓冲阈值后终端设备开始播放视频，该过程影响首次缓冲时间。在分片下载过中，一个视频分为不同品质的多个分片，分布在不同的服务器上调度，以分片为单位进行下载和播放，该过程影响卡顿事件发生。若终端设备采用“尽快下载完”当前播放分片的策略，充分利用当前实际可用带宽，边播边下，该过程影响流量模型。为降低服务器压力，终端设备采用“间歇下载”的策略即不会一直下载尚未播放的分片，若剩余播放时间大于暂停缓冲阈值则暂停下载，若剩余播放时间小于暂停缓冲阈值则重新下载，该过程影响卡顿事件发生。
为了解决上述技术问题，相关技术中通常采用内容分发网络(Content Delivery  Network，CDN)、实时消息传输协议(Real-Time messaging protocol，RTMP)、基于超文本传输协议(Hypertext Transfer Protocol，HTTP)的动态自适应流媒体(Dynamic Adaptive Streaming over HTTP，DASH)协议等技术。其中，RTMP为直播场景下音视频播放的传输协议。DASH协议为点播场景下音视频播放的主要协议。服务器存储多种清晰度的编码，根据网络情况动态选择。在一个示意性的例子中，如图2所示，RTMP的执行流程包括但不限于以下几个步骤。步骤201，终端设备发送连接请求至服务器；步骤202，服务器根据自身的带宽大小确定发送内容消息的通信窗口大小，发送通信窗口大小发送至终端设备；步骤203，服务器发送设置的带宽信息至终端设备；步骤204，终端设备根据当前接入网络的网络带宽和服务器发送的带宽信息确定协商的通信窗口大小，发送协商的通信窗口大小至服务器，从而协商完成两者之间传输数据的通信窗口大小；步骤205，终端设备发起音视频播放请求至服务器；步骤206，服务器发送内容块大小的确认消息至终端设备；步骤207，服务器发送播放操作执行成功消息至终端设备；步骤208，服务器发送音视频数据至终端设备。
上述的音视频传输技术中，服务器在终端设备发起连接请求后，通常需要至少2～3个RTT进行网络带宽探测。
而本申请实施例提供了一种音视频的处理方法、装置及存储介质，在终端设备向服务器发送音视频播放请求时，主动上报用于指示终端设备当前的接入网络的网络质量情况的网络质量信息，避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况，从而缩短了音视频的起播时延，保证了音视频的播放效果。
首先，对本申请涉及的应用场景进行介绍。
请参考图3，其示出了本申请一个示例性实施例提供的音视频的处理***的结构示意图。该***包括终端设备120和服务器140。
终端设备120中运行有音视频客户端。音视频客户端是用于播放音视频的软件应用，用户可以通过音视频客户端播放音视频。
终端设备120用于通过音视频客户端向服务器140发送音视频播放请求，接收服务器140返回的音视频数据，在音视频客户端内完成对音视频的播放。比如，终端设备120为手机，车载终端，平板电脑，电子书阅读器，动态影像专家压缩标准音频层面3(Moving Picture Experts Group Audio Layer III，MP3)播放器，动态影像专家压缩标准音频层面4(Moving Picture Experts Group Audio Layer IV，MP4)播放器，笔记本电脑，膝上型便携计算机和台式计算机等。本申请实施例对终端设备120的类型不加以限定。
可选的，终端设备120和服务器140之间建立有通信连接，该通信连接可以是有线网络或无线网络。
该服务器140也称媒体内容服务器，用于在接收到终端设备120发送的音视频播放请求后，返回音视频数据。比如，该服务器140为CDN服务器。
可选地，服务器140为音频服务器或者视频服务器。
在本申请实施例中，终端设备120用于接收播放指令，播放指令用于指示开始播放目标音视频；根据播放指令，向服务器140发送携带有网络质量信息的音视频播放 请求，网络质量信息用于指示终端设备120当前的接入网络的网络质量情况，音视频播放请求用于指示服务器140返回目标音视频的音视频数据；下载并播放目标音视频的音视频数据。
下面，采用几个示例性实施例对本申请实施例提供的音视频的处理方法进行介绍。
请参考图4，其示出了本申请一个示例性实施例提供的音视频的处理方法的流程图，本实施例以该方法用于图3所示的终端设备中来举例说明。该方法包括但不限于以下几个步骤。
步骤401，终端设备接收播放指令，播放指令用于指示开始播放目标音视频。
可选的，终端设备显示音视频客户端的用户界面，用户界面中包括目标音视频的播放控件。终端设备在接收到作用于播放控件上的用户操作信号即接收到播放指令的情况下，执行步骤402。
可选的，音视频客户端的用户界面中显示有多个音视频各自对应的播放控件。目标音视频为多个音视频中的任意一个。
步骤402，终端设备根据播放指令，向服务器发送携带有网络质量信息的音视频播放请求。
其中，网络质量信息用于指示终端设备当前的接入网络的网络质量情况，音视频播放请求用于指示服务器返回目标音视频的音视频数据。
可选的，终端设备在接收到播放指令之前，获取终端设备当前的接入网络的网络质量信息。终端设备在接收到播放指令后向服务器发送音视频播放请求，该音视频播放请求携带有网络质量信息。即在终端设备向服务器发送音视频播放请求的时候，根据***信息和历史信息主动上报网络质量信息。
其中，网络质量信息用于指示终端设备当前的接入网络的网络质量情况。网络质量信息可以包括终端设备当前的接入网络的网络带宽。网络质量信息还可以包括接入网络当前的信号强度和/或网络时延。
可选的，音视频播放请求携带有目标音视频的标识，音视频播放请求用于指示服务器返回与目标音视频的标识对应的音视频数据。
步骤403，服务器接收终端设备发送的携带有网络质量信息的音视频播放请求。
其中，网络质量信息用于指示终端设备当前的接入网络的网络质量情况。
服务器接收到终端设备发送的音视频播放请求后，获取音视频播放请求中携带的网络质量信息。
步骤404，服务器根据音视频播放请求，向终端设备返回目标音视频的音视频数据。
可选地，服务器根据网络质量信息确定通信窗口大小和播放码率；根据通信窗口大小和播放码率，向终端设备返回目标音视频的音视频数据。
可选的，音视频数据包括至少一个音视频分片，服务器向终端设备依次返回目标音视频的至少一个音视频分片。
步骤405，终端设备下载并播放目标音视频的音视频数据。
可选地，终端设备接收服务器依次返回的目标音视频的至少一个音视频分片。对 于每个音视频分片，终端设备接收到服务器返回的音视频分片后下载并播放该音视频分片。
综上所述，本申请实施例通过终端设备接收到目标音视频的播放指令后，向服务器发送携带有网络质量信息的音视频播放请求，音视频播放请求用于指示服务器返回目标音视频的音视频数据；终端设备下载并播放服务器返回的目标音视频的音视频数据；即在终端设备向服务器发送音视频播放请求时，主动上报用于指示终端设备当前的接入网络的网络质量情况的网络质量信息，避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况，从而缩短了音视频的起播时延，保证了音视频的播放效果。
相关技术中，如图5所示，发送数据的控制端在服务器侧，但是网络瓶颈往往在终端设备侧，这导致如下两个方面的问题：一方面，终端设备侧的网络质量情况反馈慢。比如，步骤501，服务器在接收到终端设备发送的连接请求后，向终端设备发送初始报文；步骤502，终端设备反馈ACK消息；步骤503，服务器向终端设备发送第二报文，第二报文的报文大小大于初始报文的报文大小；步骤504，终端设备反馈ACK消息。按照上述方法服务器进行网络带宽探测，通常需要至少2～3个RTT才能探测到网络带宽。另一方面，终端设备与服务器之间的链路不对称。比如，步骤505，服务器在探测到终端设备当前的接入网络的网络带宽后，根据网络带宽调整通信窗口大小和音视频的播放码率。步骤506，服务器下发音视频分片。终端设备实际收到音视频分片后会返回ACK消息至服务器。步骤507，若服务器没有收到终端设备发送的ACK消息，则认为网络受损，立刻降低音视频的播放码率，按照降低后的播放码率发送后续的音视频分片。若服务器没有收到的ACK消息超过预设个数，则终止发送数据。从而导致音视频的播放界面画质下降或者音视频播放过程中出现卡顿。
为了解决音视频传输流程中，网络带宽探测效率低下以及网络出现抖动情况下音视频的播放界面画质下降、卡顿的问题，本申请实施例提供了一种音视频的处理方法，包括但不限于如下两个阶段，如图6所示：第一个阶段即初始建链阶段，步骤601，终端设备通过音视频客户端发送连接请求至服务器；步骤602，服务器返回ACK消息至终端设备。步骤603，终端设备通过音视频客户端发送携带有网络质量信息的音视频播放请求至服务器，该网络质量信息包括终端设备当前的接入网络的网络带宽；步骤604，服务器根据音视频播放请求中的网络带宽，确定合适的通信窗口大小和播放码率。从而避免了相关技术中服务器与终端设备之间多次交互进行网络带宽探测的情况。第二阶段即数据传输阶段，步骤605，服务器根据确定出的通信窗口大小和播放码率，依次向终端设备返回目标音视频的至少一个音视频分片。步骤606，终端设备在接收到音视频分片后，向服务器发送多个ACK消息，确保被服务器接收到，避免相关技术中服务器因未接收到ACK消息而降低播放码率，使得画质下降，甚至降低通信窗口大小，导致卡顿的情况。步骤607，服务器在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息。
请参考图7，其示出了本申请另一个示例性实施例提供的音视频的处理方法的流程图，本实施例以该方法用于图3所示的终端设备中来举例说明。该方法包括但不限 于以下几个步骤。
步骤701，终端设备获取当前的接入网络的网络质量信息。
终端设备在接收到播放指令之前，提取获取当前的接入网络的网络质量信息。
可选的，当音视频客户端处于前台运行时，终端设备实时或者每隔预设时间段或者在接收到预设触发信号时获取网络质量信息。
示意性的，预设时间间隔是默认设置的，或者是自定义设置的。
示意性的，预设触发信号为作用于音视频客户端的用户界面中的用户操作信号。预设触发信号可以为用户界面切换信号，还可以为用户界面刷新信号。预设触发信号包括点击操作信号、滑动操作信号、按压操作信号、长按操作信号中的任意一种或多种的组合。本申请实施例对此不加以限定。
步骤702，终端设备根据接收到的播放指令，向服务器发送携带有网络质量信息的音视频播放请求。
终端设备在接收到播放指令后，向服务器发送携带有网络质量信息的音视频播放请求。
需要说明的是，相关细节可参考上述实施例中的相关描述，在此不再赘述。
步骤703，服务器根据接入网络的网络带宽和服务器的网络带宽，确定通信窗口大小和播放码率。
服务器接收终端设备发送的音视频播放请求，获取音视频播放请求中携带的网络质量信息。其中，网络质量信息包括终端设备当前的接入网络的网络带宽。可选地，网络质量信息还包括接入网络当前的信号强度和/或网络时延。
可选地，服务器根据接入网络的网络带宽和服务器的网络带宽，确定目标网络带宽；根据预设对应关系，确定与该目标网络带宽对应的通信窗口大小和播放码率，预设对应关系包括网络带宽、通信窗口大小和播放码率之间的对应关系。本申请实施例对根据网络带宽确定通信窗口大小和播放码率的方式不加以限定。
可选地，服务器根据接入网络的网络带宽和服务器的网络带宽，确定初始的通信窗口大小和播放码率，以便保证目标音视频的第一个音视频分片尽快发送至终端设备。
步骤704，服务器根据通信窗口大小和播放码率，向终端设备返回目标音视频的音视频分片。
服务器按照确定出的通信窗口大小和播放码率，依次向终端设备返回目标音视频的至少一个音视频分片。
可选地，服务器按照确定出的初始的通信窗口大小和播放码率，向终端设备返回目标音视频的第一个音视频分片。服务器在接收到终端设备返回的该音视频分片对应的ACK消息后，继续向终端设备返回目标音视频的第二个音视频分片。以此类推，不再赘述。
步骤705，终端设备在接收到音视频分片后，向服务器发送多个ACK消息。
其中，ACK消息用于指示终端设备已成功接收到音视频分片。
为防止ACK消息因为网络抖动导致服务器未收到，终端设备在接收到音视频分片后，向服务器发送多个ACK消息，其中音视频分片为目标音视频的至少一个音视频分片中的任意一个。
可选地，终端设备在接收到音视频分片后，根据采集的接入网络当前的信号强度和/或网络时延，返回一个或多个ACK消息。其中，多个ACK消息为至少两个ACK消息。
可选地，当接入网络当前的信号强度和/或网络时延满足预设条件时，终端设备向服务器发送多个ACK消息；其中，预设条件包括信号强度小于预设强度阈值，和/或网络时延大于预设时延阈值。
接入网络的信号强度和/或网络时延用于指示接入网络的网络质量。接入网络的信号强度与网络质量呈正相关关系，即接入网络的信号强度越强，表示接入网络的网络质量越好。接入网络的网络时延与网络质量呈负相关关系，即接入网络的网络时延越大，表示接入网络的网络质量越差。其中，接入网络的网络时延也称RTT。
其中，预设强度阈值或者预设时延阈值为默认设置的，或者是自定义设置的。本申请实施例对此不加以限定。
在一种可能的实现方式中，终端设备在接收到音视频分片后，判断接入网络当前的信号强度和/或网络时延是否满足预设条件，若接入网络当前的信号强度和/或网络时延满足预设条件则确定待发送的ACK消息的发送个数和发送间隔，按照发送间隔依次发送多个ACK消息；若不满足预设条件则向服务器发送一个ACK消息。
可选地，ACK消息的发送个数与接入网络的信号强度呈负相关关系。即终端设备当前的接入网络的信号强度越弱，则ACK消息的发送个数越大。示意性的，ACK消息的发送个数设置有最小值和最大值。比如，最小值为1，最大值为3。本申请实施例对此不加以限定。
可选地，发送间隔为依次发送的两个ACK消息之间的时间间隔，多个发送间隔可以是相同的，也可以是不同的。本申请实施例对此不加以限定。
在另一种可能的实现方式中，终端设备在接收到音视频分片后，向服务器发送ACK消息，并判断接入网络当前的信号强度和/或网络时延是否满足预设条件，若接入网络当前的信号强度和/或网络时延满足预设条件，则终端设备确定待发送的ACK消息的发送个数。终端设备实时检测接入网络当前的信号强度，当信号强度大于预设强度阈值时再次发送ACK消息。
需要说明的是，本申请实施例对多个ACK消息的发送方式不加以限定。
步骤706，服务器在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息。
为了避免误操作，服务器在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息，重新估算网络时延。服务器在接收到音视频分片对应的ACK消息后，保持通信窗口大小和播放码率不变，继续执行根据通信窗口大小和播放码率，向终端设备返回目标音视频的音视频分片的步骤。
在一个示意性的例子中，以终端设备为手机，服务器为视频服务器为例，手机通过无线上网(英文：Wi-Fi)方式接入家庭网络A，启动K视频客户端，根据当前接入的家庭网络A，探测到当前家庭网络A的带宽是50Mbps，在发起视频“XX烦恼”对应的视频播放请求时，通知视频服务器当前手机所接入的家庭网络A的带宽是50Mbps。视频服务器根据自身的带宽和手机当前的家庭网络A的带宽，选择最佳的视频播放码 率为1080p，按照15Mbps进行播放。由于视频是分片播放，首分片假设是2s时长，约30Mb大小，视频服务器到手机的网络时延RTT约为100ms，则手机在接收到“播放”按钮的点击操作至开始播放视频的时长(即起播时延)约为100ms，即100ms后就可以开始播放视频“XX烦恼”。而在相关技术中，由于前期至少2～3个RTT进行网络带宽探测，已经消耗200ms，加上传输控制协议(Transmission Control Protocol，TCP)慢启动本身的特性，还需要额外3～5个RTT才完成正常内容块大小的确认，此时起播时延至少在500ms以上。
在视频播放过程中，若用户携带手机走动，走到信号弱的位置，考虑障碍物对Wi-Fi的影响，可能会出现数据包丢包的情况，导致视频服务器向终端设备推送视频分片后，未接收到终端设备的ACK消息，但若用户继续走动到信号较强的位置，此时，该视频片段尚未播放完，终端设备检测到信号强度高于预设强度阈值则再次向视频服务器发送该视频片段对应的ACK消息，视频服务器获取到该ACK消息后，正常推送后续的1080p播放码率的视频分片。而在相关技术中，视频服务器在未收到ACK消息后直接降低视频的播放码率，导致画质下降，并且由于播放码率变化，可能还会导致时间延迟，出现视频播放卡顿的现象。
基于上述的例子可以看出，本申请实施例提供的音视频的处理方法，对音视频播放场景中的关键指标进行了优化：
1、启播时延，即从接收到播放指令到音视频开始播放的时长。
该指标与终端设备当前的接入网络的网络带宽、网络时延和音视频的播放码率呈相关关系。比如，待播放视频为1080p视频，网络带宽约需要15Mbps，视频的第一个视频分片为2s，该视频分片的大小为30Mb；用户家庭网络带宽为50Mb，时延为100ms。
在相关技术中，采用RTMP/HTTP DASH机制，即利用TCP协议进行传输：由于慢启动的原因，需要500ms才能加载第一个片段：第一个RTT只能传输15KB，第二个RTT可以传输30KB，第三个RTT可以传输60KB，第四个RTT可以传输120KB；第5个RTT可以传输240KB，即：15+30+60+120+240＝465KB＝3.7MKb。
而在本申请实施例中，终端设备主动上报接入网络的网络带宽为50Mb，服务器通过30Mb进行传输，在一个RTT就可以在终端设备上下载并播放音视频的第一个音视频分片。即将起播时延从500ms降到100ms。
2、网络丢包率，即所丢失的数据包数量占所发送数据包的比率。
丢包会影响音视频播放的连续性，若网络出现丢包，则音视频播放过程中可能卡顿。比如，网络丢包率为5％。终端设备采用单ACK消息进行响应的丢包率为5％，丢包后，服务器降低音视频的播放码率，影响音视频的传输吞吐，即音视频播放过程中发生卡顿的概率约为5％。
而在本申请实施例中，终端设备采用多个ACK消息进行响应，比如采用两个ACK消息进行响应的丢包率为0.25％，即仅有0.25％的概率会导致服务器降低音视频的播放码率，即音视频播放过程中发生卡顿的概率仅为0.25％。
综上所述，本申请实施例提供的音视频的处理方法，在一方面，终端设备主动上报网络质量信息：终端设备向服务器发送音视频播放请求，该音视频播放请求携带有终端设备当前的接入网络的网络质量信息，网络质量信息包括网络带宽，避免了相关 技术中网络质量信息需要服务器经过多个RTT探测才能获取的情况。在另一方面，实现了音视频播放的快速启动：服务器根据终端设备主动上报的网络质量信息确定通信窗口大小和播放码率，根据通信窗口大小和播放码率返回目标音视频的音视频数据，尽量保障目标音视频的第一个音视频分片尽快发送至终端设备，跳过了相关技术中的TCP协议的慢启动过程。在另一方面，实现了接入网络拥塞处理：为了防止ACK消息丢包，终端设备在收到音视频分片后，通过发送多个ACK消息，来避免服务器由于未收到ACK消息而进行降速的情况，其中ACK消息的发送个数与接入网络的信号强度呈负相关关系，即接入网络的信号强度越高，ACK消息的冗余量越少；只有在多个ACK消息同时丢包的情况下，服务器才会主动下降播放码率，这种方法极大地降低了音视频播放过程发生卡顿的情况；在另一方面，实现了ACK消息的去冗余处理：服务器在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息，重新估算RTT，只要服务器收到某一个ACK消息，则说明即便上行链路出现了某些问题，当前的下行链路是正常的，这种情况下服务器无需降低音视频的播放码率，可以仍然按照当前的通信窗口大小和播放码率向终端设备返回下一个音视频分片，而只有在多个ACK消息均没有收到的情况下，服务器才会降低音视频的播放码率，按照降低后的播放码率发送后续的音视频分片。
请参考图8，其示出了本申请一个示例性实施例提供的音视频的处理装置的框图。该音视频的处理装置可以通过软件、硬件或者两者的结合实现成为图3所示的终端设备的全部或者一部分。该音视频的处理装置可以包括：接收单元810、发送单元820和处理单元830。
接收单元810，用于接收播放指令，播放指令用于指示开始播放目标音视频；
发送单元820，用于根据播放指令，向服务器发送携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况，音视频播放请求用于指示服务器返回目标音视频的音视频数据；
处理单元830，下载并播放目标音视频的音视频数据。
在一种可能的实现方式中，网络质量信息包括终端设备当前的接入网络的网络带宽。
在另一种可能的实现方式中，音视频数据包括至少一个音视频分片，该装置还包括：
发送单元820，还用于在接收到音视频分片后，向服务器发送多个ACK消息，ACK消息用于指示终端设备已成功接收到音视频分片。
在另一种可能的实现方式中，发送单元820，还用于在接收到音视频分片后，当接入网络当前的信号强度和/或网络时延满足预设条件时，向服务器发送多个ACK消息；
其中，预设条件包括信号强度小于预设强度阈值，和/或网络时延大于预设时延阈值。
在另一种可能的实现方式中，ACK消息的发送个数与接入网络的信号强度呈负相关关系。
需要说明的是，上述实施例提供的装置在实现其功能时，仅以上述各个功能模块的划分进行举例说明，实际应用中，可以根据实际需要而将上述功能分配由不同的功能模块完成，即将设备的内容结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
请参考图9，其示出了本申请一个示例性实施例提供的音视频的处理装置的框图。该音视频的处理装置可以通过软件、硬件或者两者的结合实现成为图3所示的服务器的全部或者一部分。该音视频的处理装置可以包括：接收单元910和发送单元920。
接收单元910，用于接收终端设备发送的携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况；
发送单元920，用于根据音视频播放请求，向终端设备返回目标音视频的音视频数据。
在一种可能的实现方式中，网络质量信息包括接入网络的网络带宽，发送单元920，还用于：
根据接入网络的网络带宽和服务器的网络带宽，确定通信窗口大小和播放码率；
根据通信窗口大小和播放码率，向终端设备返回目标音视频的音视频数据。
在另一种可能的实现方式中，音视频数据包括至少一个音视频分片，该装置还包括：
接收单元910，还用于接收终端设备发送的音视频分片对应的多个ACK消息，ACK消息用于指示终端设备已成功接收到音视频分片。
在另一种可能的实现方式中，该装置还包括：处理单元；
处理单元，用于在接收到同一个音视频分片对应的多个ACK消息的情况下，去除重复的ACK消息。
需要说明的是，上述实施例提供的装置在实现其功能时，仅以上述各个功能模块的划分进行举例说明，实际应用中，可以根据实际需要而将上述功能分配由不同的功能模块完成，即将设备的内容结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
请参考图10，其示出了本申请一个实施例提供的终端设备的结构示意图。该终端设备包括中央处理器(Central Processing Unit，CPU)1010、存储器1020、网络接口1030。
中央处理器1010包括一个或者一个以上处理核心。中央处理器1010用于执行终端设备的各种功能应用以及用于进行数据处理。
终端设备通常包括多个网络接口1030。
存储器1020通过总线与中央处理器1010相连。存储器1020用于存储指令，处理 器1010通过执行存储器1020中存储的指令实现上述由终端设备执行的音视频的处理方法。
存储器1020可存储操作***1021和至少一个功能所需的应用程序模块1022。操作***1021包括LINUX操作***、Unix操作***、Windows操作***中的至少一种。
可选地，应用程序模块1022包括接收单元、发送单元和处理单元以及其他用于实现上述音视频的处理方法的单元等。
接收单元，用于接收播放指令，播放指令用于指示开始播放目标音视频；
发送单元，用于根据播放指令，向服务器发送携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况，音视频播放请求用于指示服务器返回目标音视频的音视频数据；
处理单元，下载并播放目标音视频的音视频数据。
可选地，存储器1020可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
请参考图11，其示出了本申请一个实施例提供的服务器的结构示意图。该服务器包括CPU1110、存储器1120、网络接口1130。
中央处理器1110包括一个或者一个以上处理核心。中央处理器1110用于执行服务器的各种功能应用以及用于进行数据处理。
服务器通常包括多个网络接口1130。
存储器1120通过总线与中央处理器1110相连。存储器1120用于存储指令，处理器1110通过执行存储器1120中存储的指令实现上述由服务器执行的音视频的处理方法。
存储器1120可存储操作***1121和至少一个功能所需的应用程序模块1122。操作***1121包括LINUX操作***、Unix操作***、Windows操作***中的至少一种。
可选地，应用程序模块1122包括接收单元、发送单元以及其他用于实现上述音视频的处理方法的单元等。
接收单元，用于接收终端设备发送的携带有网络质量信息的音视频播放请求，网络质量信息用于指示终端设备当前的接入网络的网络质量情况；
发送单元，用于根据音视频播放请求，向终端设备返回目标音视频的音视频数据。
可选地，存储器1120可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
本申请的实施例提供了一种终端设备，该终端设备包括：处理器以及用于存储处理器可执行指令的存储器；其中，处理器被配置为执行指令时实现上述终端设备侧执行的方法。
本申请的实施例提供了一种服务器，该服务器包括：处理器以及用于存储处理器可执行指令的存储器；其中，处理器被配置为执行指令时实现上述服务器侧执行的方法。
本申请的实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当计算机可读代码在电子设备的处理器中运行时，电子设备中的处理器执行上述方法。
本申请的实施例提供了一种非易失性计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令被处理器执行时实现上述方法。
计算机可读存储介质可以是保持和存储由指令执行设备使用的指令的有形设备。比如，计算机可读存储介质例包括但不限于：电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(Random Access Memory，RAM)、只读存储器(Read Only Memory，ROM)、可擦式可编程只读存储器(Electrically Programmable Read-Only-Memory，EPROM或闪存)、静态随机存取存储器(Static Random-Access Memory，SRAM)、便携式压缩盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、数字多功能盘(Digital Video Disc，DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。
这里所描述的计算机可读程序指令或代码可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本申请操作的计算机程序指令可以是汇编指令、指令集架构(Instruction Set Architecture，ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或可编程逻辑阵列(Programmable Logic Array，PLA)，该电子电路可以执行计算机可读程序指令，从而实现本申请的各个方面。
这里参照根据本申请实施例的方法、装置(***)和计算机程序产品的流程图和/ 或框图描述了本申请的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本申请的多个实施例的装置、***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。
也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行相应的功能或动作的硬件(例如电路或ASIC(Application Specific Integrated Circuit，专用集成电路))来实现，或者可以用硬件和软件的组合，如固件等来实现。
尽管在此结合各实施例对本申请进行了描述，然而，在实施所要求保护的本申请过程中，本领域技术人员通过查看所述附图、公开内容、以及所附权利要求书，可理解并实现所述公开实施例的其它变化。在权利要求中，“包括”(comprising)一词不排除其他组成部分或步骤，“一”或“一个”不排除多个的情况。单个处理器或其它单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施，但这并不表示这些措施不能组合起来产生良好的效果。
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (11)

  1. 一种音视频的处理方法，其特征在于，用于终端设备中，所述方法包括：
    接收播放指令，所述播放指令用于指示开始播放目标音视频；
    根据所述播放指令，向服务器发送携带有网络质量信息的音视频播放请求，所述网络质量信息用于指示所述终端设备当前的接入网络的网络质量情况，所述音视频播放请求用于指示所述服务器返回所述目标音视频的音视频数据；
    下载并播放所述目标音视频的音视频数据。
  2. 根据权利要求1所述的方法，其特征在于，所述网络质量信息包括所述终端设备当前的所述接入网络的网络带宽。
  3. 根据权利要求1所述的方法，其特征在于，所述音视频数据包括至少一个音视频分片，所述方法还包括：
    在接收到所述音视频分片后，向所述服务器发送多个确认应答ACK消息，所述ACK消息用于指示所述终端设备已成功接收到所述音视频分片。
  4. 根据权利要求3所述的方法，其特征在于，所述在接收到所述音视频分片后，向所述服务器发送多个确认应答ACK消息，包括：
    在接收到所述音视频分片后，当所述接入网络当前的信号强度和/或网络时延满足预设条件时，向所述服务器发送多个所述ACK消息；
    其中，所述预设条件包括所述信号强度小于预设强度阈值，和/或所述网络时延大于预设时延阈值。
  5. 根据权利要求3或4所述的方法，其特征在于，
    所述ACK消息的发送个数与所述接入网络的信号强度呈负相关关系。
  6. 一种音视频的处理方法，其特征在于，用于服务器中，所述方法包括：
    接收终端设备发送的携带有网络质量信息的音视频播放请求，所述网络质量信息用于指示所述终端设备当前的接入网络的网络质量情况；
    根据所述音视频播放请求，向所述终端设备返回目标音视频的音视频数据。
  7. 根据权利要求6所述的方法，其特征在于，所述网络质量信息包括所述接入网络的网络带宽，所述根据所述音视频播放请求，向所述终端设备返回目标音视频的音视频数据，包括：
    根据所述接入网络的网络带宽和所述服务器的网络带宽，确定通信窗口大小和播放码率；
    根据所述通信窗口大小和所述播放码率，向所述终端设备返回所述目标音视频的所述音视频数据。
  8. 根据权利要求6所述的方法，其特征在于，所述音视频数据包括至少一个音视频分片，所述方法还包括：
    接收所述终端设备发送的所述音视频分片对应的多个ACK消息，所述ACK消息用于指示所述终端设备已成功接收到所述音视频分片。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    在接收到同一个所述音视频分片对应的多个所述ACK消息的情况下，去除重复的所述ACK消息。
  10. 一种音视频的处理装置，其特征在于，所述装置包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为执行所述指令时实现权利要求1-5任意一项所述的方法，或者实现权利要求6-9任意一项所述的方法。
  11. 一种非易失性计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1-5中任意一项所述的方法，或者，实现权利要求6-9任意一项所述的方法。
PCT/CN2021/131226 2020-11-18 2021-11-17 音视频的处理方法、装置及存储介质 WO2022105798A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011292718.4A CN114584833B (zh) 2020-11-18 2020-11-18 音视频的处理方法、装置及存储介质
CN202011292718.4 2020-11-18

Publications (1)

Publication Number Publication Date
WO2022105798A1 true WO2022105798A1 (zh) 2022-05-27

Family

ID=81708362

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131226 WO2022105798A1 (zh) 2020-11-18 2021-11-17 音视频的处理方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN114584833B (zh)
WO (1) WO2022105798A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314733A (zh) * 2022-08-05 2022-11-08 京东方智慧物联科技有限公司 数据显示***、方法、电子设备及存储介质
CN115396732A (zh) * 2022-08-11 2022-11-25 深圳海翼智新科技有限公司 音视频数据包的传输方法、装置、电子设备及存储介质
CN116033235A (zh) * 2022-12-13 2023-04-28 北京百度网讯科技有限公司 数据传输方法、数字人生产设备以及数字人显示设备
CN117579874B (zh) * 2024-01-16 2024-04-05 腾讯科技（深圳）有限公司 音视频资源的传输方法、装置、服务器及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114933220B (zh) * 2022-06-17 2024-03-15 广东美房智高机器人有限公司 机器人乘梯方法、装置、服务器、嵌入式设备及存储介质
CN117440177A (zh) * 2022-07-12 2024-01-23 腾讯科技（深圳）有限公司 视频流传输的控制方法及装置、设备、介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014301A (zh) * 2010-11-26 2011-04-13 优视科技有限公司 一种视频播放方法、***及服务器
CN102595204A (zh) * 2012-02-28 2012-07-18 华为终端有限公司 一种流媒体传输方法、设备及***
CN103402077A (zh) * 2013-07-24 2013-11-20 佳都新太科技股份有限公司 一种公网ip网内动态调整码流速率的视音频传输策略方法
US20170078182A1 (en) * 2015-09-14 2017-03-16 Dynatrace Llc Method And System For Automated Injection Of Process Type Specific In-Process Agents On Process Startup
CN107634881A (zh) * 2017-09-28 2018-01-26 苏州蜗牛数字科技股份有限公司 一种网络或视频业务探测***及方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101404622B (zh) * 2008-11-07 2011-03-23 重庆邮电大学 基于多径负载均衡的无线互联网拥塞控制方法及控制器
CN102468941B (zh) * 2010-11-18 2014-07-30 华为技术有限公司 网络丢包处理方法及装置
WO2016172818A1 (zh) * 2015-04-27 2016-11-03 华为技术有限公司 应答消息传输方法和网络设备
CN107071518A (zh) * 2016-09-05 2017-08-18 北京奥鹏远程教育中心有限公司 自适应移动终端学习的视频播放方法和***
CN106559715B (zh) * 2016-11-23 2019-08-06 中国联合网络通信集团有限公司 移动网络视频传输优化方法及装置
CN109729396B (zh) * 2017-10-31 2022-03-11 华为技术有限公司 视频分片数据传输方法和装置
CN109922507A (zh) * 2019-01-26 2019-06-21 成都鑫芯电子科技有限公司 一种基于低功耗传感器的无线传输***及方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014301A (zh) * 2010-11-26 2011-04-13 优视科技有限公司 一种视频播放方法、***及服务器
CN102595204A (zh) * 2012-02-28 2012-07-18 华为终端有限公司 一种流媒体传输方法、设备及***
CN103402077A (zh) * 2013-07-24 2013-11-20 佳都新太科技股份有限公司 一种公网ip网内动态调整码流速率的视音频传输策略方法
US20170078182A1 (en) * 2015-09-14 2017-03-16 Dynatrace Llc Method And System For Automated Injection Of Process Type Specific In-Process Agents On Process Startup
CN107634881A (zh) * 2017-09-28 2018-01-26 苏州蜗牛数字科技股份有限公司 一种网络或视频业务探测***及方法

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314733A (zh) * 2022-08-05 2022-11-08 京东方智慧物联科技有限公司 数据显示***、方法、电子设备及存储介质
CN115396732A (zh) * 2022-08-11 2022-11-25 深圳海翼智新科技有限公司 音视频数据包的传输方法、装置、电子设备及存储介质
CN115396732B (zh) * 2022-08-11 2024-02-02 深圳海翼智新科技有限公司 音视频数据包的传输方法、装置、电子设备及存储介质
CN116033235A (zh) * 2022-12-13 2023-04-28 北京百度网讯科技有限公司 数据传输方法、数字人生产设备以及数字人显示设备
CN116033235B (zh) * 2022-12-13 2024-03-19 北京百度网讯科技有限公司 数据传输方法、数字人生产设备以及数字人显示设备
CN117579874B (zh) * 2024-01-16 2024-04-05 腾讯科技（深圳）有限公司 音视频资源的传输方法、装置、服务器及存储介质

Also Published As

Publication number Publication date
CN114584833B (zh) 2024-05-17
CN114584833A (zh) 2022-06-03

Similar Documents

Publication Publication Date Title
WO2022105798A1 (zh) 音视频的处理方法、装置及存储介质
CN111135569B (zh) 云游戏处理方法、装置、存储介质与电子设备
CN111628847B (zh) 数据传输方法及装置
US9794311B2 (en) Transport accelerator implementing extended transmission control functionality
US20220272402A1 (en) Video stream playing method, system, terminal and storage medium
EP3322145A1 (en) Method, server side and system for computing bandwidth of network transmission of streaming media
US11792130B2 (en) Audio/video communication method, terminal, server, computer device, and storage medium
US9420022B2 (en) Media requests to counter latency and minimize network bursts
US20120047230A1 (en) Client-initiated management controls for streaming applications
WO2018121742A1 (zh) 一种流数据的传输方法和装置
US10334287B2 (en) Digital data streaming using server driven adaptive bitrate
US20230143627A1 (en) Method and server for audio and/or video content delivery
CN110830460A (zh) 一种连接建立方法、装置、电子设备及存储介质
US9350484B2 (en) Transport accelerator implementing selective utilization of redundant encoded content data functionality
CN113726817B (zh) 一种流媒体数据的传输方法、装置及介质
CN113301398B (zh) 信息处理方法及***、服务端设备、客户端设备
CN115333677A (zh) 云业务处理方法、***、装置、设备及存储介质
WO2024051426A1 (zh) 视频流码率调整方法、装置、计算机设备和存储介质
TW201501527A (zh) 用戶終端機用以檢索多媒體內容之內容組件之方法
EP4002793A1 (en) Method and controller for audio and/or video content delivery
CN116319701A (zh) 用于音视频的流传输方法及装置、电子设备和介质
CN116132721A (zh) 一种基于p2p的弱网环境下车载投屏方法及***
CN114501083A (zh) 码率调节方法、装置、设备、介质及程序产品
CN117222457A (zh) 涉及云游戏中的体验质量改善的方法、装置和***
CN115604194A (zh) 视频传输速率的段内调整

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21893945

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21893945

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26/09/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21893945

Country of ref document: EP

Kind code of ref document: A1