WO2021143344A1 - 码率决策模型训练方法以及电子设备 - Google Patents

码率决策模型训练方法以及电子设备 Download PDF

Info

Publication number
WO2021143344A1
WO2021143344A1 PCT/CN2020/129671 CN2020129671W WO2021143344A1 WO 2021143344 A1 WO2021143344 A1 WO 2021143344A1 CN 2020129671 W CN2020129671 W CN 2020129671W WO 2021143344 A1 WO2021143344 A1 WO 2021143344A1
Authority
WO
WIPO (PCT)
Prior art keywords
decision
code rate
model
moment
time
Prior art date
Application number
PCT/CN2020/129671
Other languages
English (en)
French (fr)
Inventor
周超
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Priority to EP20914063.1A priority Critical patent/EP3968648A4/en
Publication of WO2021143344A1 publication Critical patent/WO2021143344A1/zh
Priority to US17/562,687 priority patent/US20220124387A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23406—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving management of server-side video buffer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115—Selection of the code volume for a coding unit prior to coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234354—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering signal-to-noise ratio parameters, e.g. requantization
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2401—Monitoring of the client buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/251—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
    • H04N21/6379—Control signals issued by the client directed to the server or network components directed to server directed to encoder, e.g. for requesting a lower encoding rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

  • the present disclosure relates to the field of live video broadcasting, and in particular to a method for training a code rate decision model and an electronic device.
  • Network fluctuations have a huge impact on the field of live video streaming.
  • electronic devices need to adjust the video stream bit rate according to network fluctuations.
  • Video bit rate control generally adopts the following method: adjust the bit rate of the video data sent by the electronic device to keep the video duration buffered by the client within a given range, for example, to maintain the duration of the live video buffered by the client within a given range. Between 10s and 20s, when the cached live video duration is less than 10s, the bit rate of transmission will be reduced, and the clarity of the video will be reduced, so that the same video data packet carries a longer live video; when the cached live video has a duration When it is greater than 20s, the bit rate of the transmission is increased, and the definition of the video is improved, so that the same video data packet carries a live video with a shorter duration and a longer duration.
  • the present disclosure provides a method for training a code rate decision model and an electronic device.
  • the technical solutions of the present disclosure are as follows:
  • a method for training a rate decision model is provided, which is applied to electronic equipment, including:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple corresponding to multiple first decision code rates
  • the target decision code rate at the first moment is input into the simulation interactive environment to obtain second time length change information, where the second time length change information is the buffer simulation module in the simulation interactive environment at the third time The time length change information of, wherein the third moment is the next video data transmission moment of the first moment;
  • a first evaluation value is obtained, and the first evaluation value is the first evaluation value at the first time Evaluation value of target decision code rate;
  • a code rate decision method is provided, which is applied to electronic equipment, including:
  • the network throughput at the fifth moment, the first parameter change information, and the target decision code rate at the sixth moment are simultaneously input into the rate decision model, and the rate decision model outputs multiple codes corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition
  • the code rate of the video data is adjusted based on the target decision code rate at the fifth moment, and the code rate decision model is a code rate decision model trained using the code rate decision model training method of claim 1.
  • a device for training a code rate decision model which includes:
  • the first probability output unit is configured to simultaneously input the network throughput at the first moment, the change information of the first time length, and the target decision code rate at the second moment into the first model, and the first model outputs more information.
  • the first target decision code rate determining unit is configured to perform the determination of the target decision code rate at the first moment, and the target decision code rate at the first moment is a first decision code rate with a first probability that meets the first target condition ;
  • the time length change information acquiring unit is configured to execute inputting the target decision code rate at the first moment into the simulation interactive environment to obtain second time length change information, where the second time length change information is the third Time length change information of the buffer simulation module in the simulation interactive environment at the moment, where the third moment is the next video data transmission moment of the first moment;
  • the evaluation value acquisition unit is configured to perform the acquisition of a first evaluation value based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time.
  • the evaluation value is the evaluation value of the target decision code rate at the first moment;
  • the model parameter update unit updates the model parameters of the first model based on the first evaluation value until any iteration process satisfies the first iteration end condition to obtain the rate decision model, and the rate decision model meets all requirements.
  • the first model obtained by the iterative process of the end condition of the first iteration is described.
  • a code rate decision device which includes:
  • the second probability output unit is configured to simultaneously input the network throughput at the fifth time, the first parameter change information, and the target decision code rate at the sixth time into the rate decision model, and output by the rate decision model Multiple third probabilities corresponding to multiple third decision code rates, wherein the sixth time is the last code rate decision time of the fifth time, and the first parameter change information is the fifth time The parameter change information of the buffer;
  • the second target decision code rate determination unit is configured to perform the determination of the target decision code rate at the fifth moment, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition ;
  • the code rate adjustment unit is configured to perform the adjustment of the code rate of the video data based on the target decision code rate at the fifth moment, and the code rate decision model is trained by the code rate decision model training device of claim 9 The bit rate decision model.
  • an electronic device including:
  • a memory for storing executable instructions of the processor
  • the processor is configured to perform the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple corresponding to multiple first decision code rates
  • the target decision code rate at the first moment is input into the simulation interactive environment to obtain second time length change information, where the second time length change information is the buffer simulation module in the simulation interactive environment at the third time The time length change information of, wherein the third moment is the next video data transmission moment of the first moment;
  • a first evaluation value is obtained, and the first evaluation value is the first evaluation value at the first time Evaluation value of target decision code rate;
  • an electronic device including:
  • a memory for storing executable instructions of the processor
  • the processor is configured to perform the following steps:
  • the network throughput at the fifth moment, the first parameter change information, and the target decision code rate at the sixth moment are simultaneously input into the rate decision model, and the rate decision model outputs multiple codes corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition
  • the bit rate of the video data is adjusted, and the bit rate decision model is a bit rate decision model trained by the electronic device in the previous aspect.
  • a storage medium is provided.
  • the electronic device can execute the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple corresponding to multiple first decision code rates
  • the target decision code rate at the first moment is a first decision code rate with a first probability that meets the first target condition
  • the target decision code rate at the first moment is input into the simulation interactive environment to obtain second time length change information, where the second time length change information is the buffer simulation module in the simulation interactive environment at the third time The time length change information of, wherein the third moment is the next video data transmission moment of the first moment;
  • a first evaluation value is obtained, and the first evaluation value is the first evaluation value at the first time Evaluation value of target decision code rate;
  • a storage medium is provided.
  • the electronic device can execute the following steps:
  • the network throughput at the fifth moment, the first parameter change information, and the target decision code rate at the sixth moment are simultaneously input into the rate decision model, and the rate decision model outputs multiple codes corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition
  • the bit rate of the video data is adjusted, and the bit rate decision model is a bit rate decision model trained using the storage medium described in the previous aspect.
  • a computer program product is provided.
  • the instructions in the computer program product are executed by the processor of the electronic device, the electronic device can execute the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple corresponding to multiple first decision code rates
  • the target decision code rate at the first moment is input into the simulation interactive environment to obtain second time length change information, where the second time length change information is the buffer simulation module in the simulation interactive environment at the third time The time length change information of, wherein the third moment is the next video data transmission moment of the first moment;
  • a first evaluation value is obtained, and the first evaluation value is the first evaluation value at the first time Evaluation value of target decision code rate;
  • a computer program product is provided.
  • the instructions in the computer program product are executed by the processor of the electronic device, the electronic device can execute the following steps:
  • the network throughput at the fifth moment, the first parameter change information, and the target decision code rate at the sixth moment are simultaneously input into the rate decision model, and the rate decision model outputs multiple codes corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition
  • the code rate of the video data is adjusted based on the target decision code rate at the fifth moment, and the code rate decision model is a code rate decision model trained by using the computer program product described in the previous aspect.
  • Figure 1 is a schematic diagram of a video data collection type.
  • Figure 2 is a schematic structural diagram of a simulated interactive environment.
  • Figure 3 is a flow chart of a method for training a rate decision model.
  • Figure 4 is a flow chart of a code rate decision method.
  • Figure 5 is a flow chart of a method for training a rate decision model.
  • Figure 6 is a schematic diagram of the structure of an Actor network.
  • Figure 7 is a schematic diagram of the structure of a Critic network.
  • Fig. 8 is a block diagram of a code rate decision model training device.
  • Figure 9 is a block diagram of a code rate decision device.
  • Fig. 10 is a structural diagram of an electronic device.
  • Fig. 11 is a structural diagram of an electronic device.
  • Fig. 12 is a graph showing an example of bandwidth change over time.
  • the bit rate is used to indicate how much information a video block of a certain length of time contains.
  • the video with a larger bit rate has higher definition under the same compression ratio.
  • the buffer is used to store the unsent video data.
  • the size of the buffer is limited. In the field of live broadcast, it is hoped to control the video data stored in the buffer to be as small as possible to ensure the real-time nature of the live broadcast.
  • Network throughput refers to the amount of data transmitted per unit time.
  • the embodiment of the present disclosure trains a code rate decision model to predict the required code rate at the next moment based on the network transmission information at the previous moment.
  • the following describes the code rate decision model training method adopted in the embodiment of the present disclosure.
  • the training method of rate decision model includes three processes: data collection, training environment creation and training.
  • the electronic device obtains relevant information used by multiple electronic devices to represent the video transmission environment during the data transmission process.
  • the relevant information includes, but is not limited to, historical network throughput W and buffer time. Length information B, historical bit rate decision R, and historical buffer time length change information ⁇ B.
  • the data collection time scale corresponding to the above-mentioned related information can be different.
  • the data collection time scale can include a long interval and a short interval, where the long interval is the time interval between two code rate decisions, and the short interval is the relative interval.
  • the time interval between two adjacent video frames of course, the length of the long interval and the short interval can be set according to actual needs, which is not limited in the embodiment of the present disclosure.
  • Bit rate decision refers to the way to adjust the current bit rate.
  • the electronic device can simultaneously collect the network throughput W of the long interval and the short interval, denoted as W L and W S respectively ; it can also collect the long interval and short interval at the same time.
  • Buffer time length information B denoted as B L and B S respectively ;
  • historical bit rate decision R can only collect long interval data, denoted as R L ;
  • historical buffer time length change information ⁇ B can only collect short interval data, Denoted as ⁇ B S.
  • the data of different time scales acquired by electronic equipment has different meanings for the code rate decision.
  • the information of short intervals can deal with the sudden situation in the code rate decision, and the information on the long interval can make the code
  • the rate decision model can better grasp the global information of the data and reduce the occurrence of wrong decision.
  • the basic model architecture adopts any kind of neural network, for example, deep deterministic policy gradient (DDPG) and asynchronous advantage actor- critic, A3C) or policy gradients (policy gradients), etc., which are not limited in the embodiment of the present disclosure.
  • DDPG deep deterministic policy gradient
  • A3C asynchronous advantage actor- critic
  • policy gradients policy gradients
  • the code rate decision model training part in the embodiment of the present disclosure may use a simulation interactive environment to interact with the model.
  • the embodiment of the present disclosure uses a simulation interaction environment to interact with the model to train the code rate decision model. Based on this, a simulation interactive environment can be created to simulate actual network throughput changes.
  • the electronic device inputs the collected records of real network throughput changes over time into the simulated interactive environment, and the simulated interactive environment sends video data according to the collected real network throughput, and obtains relevant information of the current video transmission environment.
  • the bit rate decision model outputs the corresponding bit rate decision based on the relevant information of the video transmission environment obtained from the simulated interactive environment. After that, the decision evaluation model will output the evaluation value according to the decision code rate output by the code rate decision model. Since the rate decision made by the rate decision model will affect the simulation interaction environment, the simulation interaction environment will change to a certain extent, and finally the relevant information fed back to the decision evaluation model will also change. The entire training process is the repetition of the above interaction actions.
  • the obtained price evaluation value is a reward function that is conducive to rate decision learning.
  • the structure of the simulation interactive environment is shown in FIG. 2 and includes three modules: an encoder simulation module, a buffer simulation module, and a sending simulation module.
  • the encoder simulation module is used to receive the bit rate prediction output by the bit rate decision model, and send the video data of the corresponding bit rate to the buffer simulation module.
  • the size of video data will not only be affected by the bit rate, but also by the size of each frame in the video data.
  • the encoder emulation module can encode the size of the video data to meet the bit rate constraint. Random fluctuations within a certain range, and at the same time, the code rate and video data size that conform to the actual live video need to be set on the encoder simulation module.
  • the buffer emulation module is used to receive and send video data.
  • the buffer emulation module can be implemented based on a queue with limited capacity.
  • the buffer emulation module receives the video data sent by the encoder emulation module at a certain frame interval, and at the same time sends the video data to the sending emulation module.
  • the sending simulation module is used to receive virtual network throughput, where virtual network throughput is used to simulate changes in the actual available bandwidth of the network.
  • the virtual network throughput is the change in real bandwidth collected in advance over time. record of.
  • the sending simulation module is also used to send the video data in the buffer simulation module according to the limitation of the network throughput, so as to achieve the purpose of consuming the video data of the buffer simulation module at a speed determined by the network throughput.
  • the encoder simulation module sends fixed-duration video data to the buffer simulation module each time.
  • the network throughput fluctuates over time.
  • the video data is taken from the buffer emulation module based on the network throughput, which means that the sending emulation module does not randomly take out the video data from the buffer emulation module.
  • the speed of fetching video data is determined by the current network throughput.
  • the current network throughput is 1000KB/s, which means that the sending simulation module can take out 1000KB of video data from the buffer simulation module per second. If the size of a single video data is 50KB, it is equivalent to sending the simulation module 20 video data can be fetched per second; if the size of a video data is 25KB, it is equivalent to the sending simulation module can fetch 40 video data per second.
  • the remaining capacity of the buffer emulation module changes all the time.
  • the sending emulation module will take the video data out of the buffer emulation module slowly. In this case, because the encoder emulation module The amount of video data sent by the buffer is certain, so the data stored in the buffer emulation module will increase, and accordingly, the remaining capacity of the buffer will be reduced.
  • the video data in the buffer emulation module if the video data in the buffer emulation module reaches the upper limit of its capacity, the video data is discarded according to the "first in, first out" principle.
  • the embodiments of the present disclosure provide a model training process based on a simulated interactive environment.
  • the basic idea is that the electronic device makes a bit rate decision based on the sample data through the first model, and the first model outputs a decision bit rate.
  • the electronic device inputs the decision bit rate into the simulation interactive environment, and the simulation interactive environment adjusts the transmission bit rate of the video data based on the received decision bit rate.
  • the electronic device obtains the time length change information of the buffer simulation module in the simulation interactive environment in the above process, and inputs the time length change information of the buffer simulation module, the decision bit rate and the network throughput into the second model, and the second model
  • the evaluation value is output, and the electronic device updates the model parameters of the first model based on the evaluation value.
  • the continuous interaction of the first model, the simulated interactive environment, and the second model finally results in a bit rate decision model, which has the ability to predict the decision bit rate based on the relevant information of the video transmission environment.
  • a bit rate decision model which has the ability to predict the decision bit rate based on the relevant information of the video transmission environment. Refer to 501-507 for the training process of the rate decision model.
  • Figure 3 is a flow chart of a method for training a rate decision model, applied to electronic equipment, as shown in Figure 3, including the following steps:
  • the electronic device inputs the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment into the first model at the same time, and the first model outputs multiple codes corresponding to the first decision code rate.
  • a plurality of first probabilities where the second time is the last bit rate decision time at the first time, and the first time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the first time.
  • the electronic device determines the target decision code rate at the first moment, and the target decision code rate at the first moment is a first decision code rate with a first probability of meeting the first target condition.
  • the electronic device inputs the target decision code rate at the first moment into the simulation interactive environment to obtain second time length change information, where the second time length change information is the time of the buffer simulation module in the simulation interactive environment at the third moment Length change information, where the third moment is the next video data transmission moment of the first moment.
  • the electronic device obtains a first evaluation value based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time, where the first evaluation value is an evaluation of the target decision code rate at the first time. value.
  • the electronic device updates the model parameters of the first model based on the first evaluation value until any iteration process meets the first iteration end condition to obtain a rate decision model, which is an iterative process that meets the first iteration end condition The first model obtained.
  • the simulation interactive environment further includes an encoder simulation module and a sending simulation module, inputting the target decision bit rate at the first moment into the simulation interactive environment, and obtaining second time length change information includes:
  • the target decision code rate at the first moment is input to the encoder simulation module, and the encoder simulation module sends the video data of the target decision code rate at the first moment to the buffer simulation module.
  • the video data is extracted from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the second time length change information is acquired based on the difference between the storage capacity of the video data at the first time and the third time by the buffer emulation module.
  • the network throughput includes a first network throughput and a second network throughput, where the first network throughput is the network throughput collected within a time range of two video frame intervals, and the second network throughput The quantity is the network throughput of the bit rate decision interval.
  • the time length change information of the buffer emulation module includes the first buffer time length change information and the second buffer time length change information.
  • the first buffer time length change information is the buffer collected within the time range of the two video frame intervals. Zone time length change information, the second buffer time length change information bit rate decision interval buffer time length change information.
  • obtaining the first evaluation value of the target decision code rate at the first moment includes:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the second model, and the second model outputs the first evaluation value.
  • the method further includes:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the first model to obtain the target decision code rate at the third moment.
  • the decision evaluation model is the second model obtained by the iterative process that satisfies the end condition of the second iteration, where the fourth moment is the next video data transmission moment at the third moment, and the third time length change information is the simulation at the fourth moment Time length change information of the buffer emulation module in the interactive environment.
  • the method before simultaneously inputting the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment into the first model, the method further includes:
  • sample data is input to the first model, and the first model outputs multiple second probabilities corresponding to multiple second decision code rates.
  • the sample data includes historical decision code rate and historical buffer time length Information, historical buffer time length change information, and historical network throughput.
  • the sample target code rate is determined, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the sample target code rate is input into the simulation interactive environment, and the sample time length change information of the buffer simulation module in the simulation interactive environment is obtained.
  • the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission moment are input into the second model, and the second model outputs the second evaluation value.
  • the model parameters of the first model are updated.
  • Figure 4 is a flow chart of a code rate decision method, applied to electronic equipment, as shown in Figure 4, including the following steps:
  • the electronic device inputs the network throughput at the fifth moment, the first parameter change information, and the target decision code rate at the sixth moment into the rate decision model at the same time, and the rate decision model outputs multiple third decision rate correspondences There are multiple third probabilities of, where the sixth time is the last bit rate decision time at the fifth time, and the first parameter change information is the parameter change information of the buffer at the fifth time.
  • the electronic device determines the target decision code rate at the fifth moment, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that the third target condition is met.
  • that the third probability meets the third target condition means that the third probability is the highest among a plurality of third probabilities.
  • the electronic device adjusts the code rate of the video data based on the target decision code rate at the fifth moment, and the code rate decision model is a code rate decision model trained by the above code rate decision model training method.
  • the method further includes:
  • the model parameters of the code rate decision model are updated, and the seventh time is the next video data transmission time at the fifth time.
  • Figure 5 is a flow chart of a method for training a rate decision model. As shown in Figure 5, the method is applied to an electronic device and includes the following steps:
  • the electronic device inputs the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment into the first model at the same time, and the first model outputs multiple codes corresponding to the first decision code rate.
  • a plurality of first probabilities where the second time is the last bit rate decision time at the first time, and the first time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the first time.
  • the electronic device composes a first parameter matrix by the network throughput at the first moment, the change information of the first time length, and the target decision code rate at the second moment, and inputs the first parameter matrix into the first model
  • at least one weight matrix of the first model is multiplied by the parameter matrix to obtain multiple first eigenvectors, and the multiple first eigenvectors are mapped to multiple first probabilities. For example, if the network throughput W at the first moment is 500Kbps, the target decision code rate R at the second moment is 0.7, and the first time length change information ⁇ B is 3%, the electronic device generates a one-dimensional first parameter matrix [500 , 0.7, 3] T.
  • the electronic device multiplies the weight matrix [0.2, 1, 0.3] with the first parameter matrix to obtain the first eigenvector [10, 0.7, 0.9] T , which is mapped into multiple first eigenvectors through the normalization function (SoftMax) Probability, such as [0.76, 0.05, 0.07] T , where the number in the first feature vector represents the probability of the corresponding first decision code rate.
  • SoftMax normalization function
  • the interval between the first time and the second time is set in advance, and the timing is directly switched.
  • the interval between the first time and the second time is an interval determined by the electronic device in real time, which is not limited in the embodiment of the present disclosure.
  • the first decision code rate is a multiple of code rate adjustment, or a code rate value of one. If the first decision code rate is a multiple of the code rate adjustment, then the first probability output by the code rate decision model corresponds to a different code rate adjustment multiple, such as 0.7, 0.8, 0.9, 1.0, 1.05, 1.1 and 1.15, each number It is used to indicate that the bit rate of the current video data is adjusted to the multiple of the previous moment. If among the multiple first decision bit rates, 0.7 corresponds to the highest first probability, then the electronic device uses the encoder simulation module to change the current video data The code rate is adjusted to 0.7 times of the previous moment.
  • the electronic device determines the target decision code rate at the first moment, and the target decision code rate at the first moment is a first decision code rate that has a first probability of meeting the first target condition.
  • that the first probability meets the first target condition means that the first probability is the highest among the multiple first probabilities.
  • the electronic device inputs the target decision code rate at the first moment into the simulation interactive environment to obtain second time length change information, where the second time length change information is the time of the buffer simulation module in the simulation interactive environment at the third moment Length change information, where the third moment is the next video data transmission moment of the first moment.
  • the simulation interactive environment further includes an encoder simulation module and a sending simulation module.
  • the electronic device inputs the target decision code rate at the first moment into the encoder simulation module, and the encoder simulation module sends the first code rate to the buffer simulation module.
  • the electronic device extracts video data from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the electronic device obtains the second time length change information based on the difference in the storage capacity of the video data at the first time and the third time by the buffer simulation module.
  • the electronic device simultaneously inputs the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment into the second model, and the second model outputs the first evaluation value.
  • the electronic device composes the second parameter matrix of the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment.
  • the electronic device inputs the second parameter matrix into the second model, performs operations through at least one weight matrix of the second model to obtain a plurality of second eigenvectors, and maps the plurality of second eigenvectors to the first evaluation value.
  • the electronic device For example, if the network throughput W at the third moment is 450Kbps, the target decision bit rate R at the first moment is 0.5, and the time length change information ⁇ B of the buffer simulation module at the third moment is 2%, the electronic device generates a one-dimensional The second parameter matrix [450, 0.5, 2] T. The electronic device multiplies the weight matrix [0.1, 1, 0.5] with the second parameter matrix to obtain the second eigenvector [4.5, 0.5, 1] T , which is mapped to the first evaluation value through the S-shaped growth curve (Sigmoid) , For example 0.6.
  • Sigmoid S-shaped growth curve
  • the electronic device updates the model parameters of the first model based on the first evaluation value until any iteration process meets the first iteration end condition to obtain the rate decision model, and the rate decision model is an iterative process that satisfies the first iteration end condition The first model obtained.
  • the electronic device updates at least one weight matrix of the first model based on the first evaluation value, until the function value of the loss function of the first model is lower than the target threshold, or the number of iterations reaches the target number.
  • the code rate decision model is obtained, where the target threshold and the target number of times can be set according to the actual situation, which is not limited in the embodiment of the present disclosure.
  • the electronic device inputs the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment into the first model at the same time, to obtain the target decision code rate at the third moment.
  • the method for determining the target decision code rate at the third moment by the electronic device and the method for determining the target decision code rate at the first moment belong to the same inventive concept, and will not be repeated here.
  • the electronic device updates the model parameters of the second model based on the network throughput at the fourth moment, the third time length change information, and the target decision code rate at the third moment, until any iteration process meets the second iteration end condition, and the decision is obtained Evaluation model, where the decision evaluation model is the second model obtained from the iterative process that satisfies the end conditions of the second iteration, where the fourth moment is the next video data transmission moment at the third moment, and the third time length change information is the first Time length change information of the buffer simulation module in the simulation interactive environment at four moments.
  • the electronic device inputs the sample data into the first model.
  • Model the first model outputs multiple second probabilities corresponding to multiple second decision code rates, where the sample data includes historical decision code rates, historical buffer time length information, historical buffer time length change information, and historical network throughput quantity.
  • the electronic device determines a sample target code rate, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the electronic device inputs the target code rate of the sample into the simulation interactive environment, and obtains sample time length change information of the buffer simulation module in the simulation interactive environment.
  • the electronic device inputs the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission moment into the second model, and the second model outputs the second evaluation value.
  • the electronic device updates the model parameters of the first model based on the second evaluation value.
  • that the second probability meets the second target condition means that the second probability is the highest among a plurality of second probabilities.
  • the sample data comes from the client watching the video or comes from the server.
  • the embodiment of the present disclosure does not limit the source of the sample data.
  • the electronic device can simultaneously train multiple rate decision models based on different network bandwidths, and obtain the parameters of the primary rate decision model based on the model parameters of the multiple rate decision models obtained through training.
  • the model is also a rate decision model for rate decision making during live broadcast.
  • a plurality of code rate decision models can respectively obtain a first reference number of training parameters, and the training parameters include at least relevant information representing the video transmission environment and an evaluation value corresponding to the relevant information representing the video transmission environment.
  • the electronic device sends the first reference number of training parameters to the master bit rate decision model through multiple code rate decision models, and the electronic device updates the model parameters of the master bit rate decision model based on the first reference number of training parameters.
  • the electronic device sends the updated model parameters to the multiple rate decision models through the master rate decision model.
  • the electronic device controls multiple code rate decision models to replace the model parameters with the received model parameters, and then continue to train in different simulation interactive environments.
  • the above steps are repeated until the number of times the electronic device updates the model parameters of the master bit rate decision model reaches the reference number, and the training ends.
  • the reference times can be set according to actual needs, which is not limited in the embodiment of the present disclosure.
  • the main bit rate decision model provided in the embodiments of the present disclosure can also determine the time to stop training based on the loss function of the model, which is not limited in the embodiments of the present disclosure. .
  • an asynchronous update enhancement model (actor-critic) is used as an example to illustrate the training method of the code rate decision model provided by the embodiment of the present disclosure.
  • the code rate decision model is the Actor (strategy) network
  • the decision evaluation model is the Critic (value) network.
  • the Critic network is used for buffer simulation in the simulation interactive environment based on the selection of different code rates under different network throughputs.
  • the time length change information of the module is used to output the evaluation value, and the Actor network adjusts the model parameters based on the evaluation value output by the Critic network.
  • the Critic network is based on the relevant information used to represent the video transmission environment at the current moment and the decision bit rate at the previous moment. Adjust model parameters.
  • the Critic network evaluates the decision code rate output by the Actor network
  • the Actor network uses the evaluation value of the output of the Critic network as the training target.
  • the Actor network adjusts the model parameters through the following formula (1)
  • the Critic network adjusts the model parameters through the following formula (2):
  • ⁇ a is the network parameter of the Actor
  • ⁇ a is the learning rate of the Actor network
  • ⁇ ⁇ (s t , a t ) is the code rate prediction of the Actor network
  • A(s t , a t ) is the evaluation output of the Critic network.
  • ⁇ c is the parameter of the Critic network
  • ⁇ c is the learning rate of the Critic network
  • V ⁇ (s t , ⁇ c ) is the network transmission information s t at time t and the current Critic network parameter ⁇ c under the Critic network output Evaluation value.
  • only the Actor network is in the active state when the code rate decision is made, and the Actor network and the Critic network are in the active state at the same time in the training state.
  • the last output layer of the Critic network and the Actor network are different.
  • the last output layer of the Critic network is a linear output layer without activation function; the last output layer of the Actor network is a SoftMax output layer. Except for the last output layer, the rest of the structure of the Critic network and the Actor network are the same.
  • the structure of the Actor network is shown in FIG. 6, and the structure of the Critic network is shown in FIG. 7. It should be noted that the structures of the Critic network and the Actor network can be designed based on actual conditions, which are not limited in the embodiment of the present disclosure.
  • Fig. 8 is a block diagram of a code rate decision model training device. 8, the device includes a first probability output unit 801, a first target decision code rate determination unit 802, a time length change information determination unit 803, an evaluation value acquisition unit 804, and a model parameter update unit 805.
  • the first probability output unit 801 is configured to simultaneously input the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment into the first model, and the first model outputs multiple Multiple first probabilities corresponding to the first decision code rate, where the second time is the last code rate decision time at the first time, and the first time length change information is the buffer simulation module in the simulation interactive environment at the first time The length of time change information.
  • the first target decision code rate determining unit 802 is configured to determine the target decision code rate at the first moment, and the target decision code rate at the first moment is a first decision code rate with a first probability of meeting the first target condition.
  • the time length change information acquiring unit 803 is configured to input the target decision code rate at the first moment into the simulation interaction environment to obtain second time length change information, where the second time length change information is the simulation interaction at the third time Time length change information of the buffer emulation module in the environment, where the third moment is the next video data transmission moment of the first moment.
  • the evaluation value acquisition unit 804 is configured to perform a first evaluation value based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time. The evaluation value of the target decision code rate.
  • the model parameter update unit 805 is configured to update the model parameters of the first model based on the first evaluation value until any iteration process meets the first iteration end condition, and the rate decision model is obtained, and the rate decision model meets the first The first model obtained by the iterative process of the iterative end condition.
  • the simulation interactive environment further includes an encoder simulation module and a sending simulation module
  • the time length change information acquisition unit includes:
  • the video data sending subunit is configured to execute inputting the target decision bit rate at the first moment into the encoder simulation module, and the encoder simulation module sends the video data of the target decision bit rate at the first moment to the buffer simulation module.
  • the video data extraction subunit is configured to extract video data from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the time length change information acquisition subunit is configured to perform the acquisition of the second time length change information based on the difference in the storage capacity of the video data at the first time and the third time by the buffer emulation module.
  • the network throughput includes a first network throughput and a second network throughput, where the first network throughput is the network throughput collected within a time range of two video frame intervals, and the second network throughput The quantity is the network throughput of the bit rate decision interval.
  • the time length change information of the buffer emulation module includes the first buffer time length change information and the second buffer time length change information.
  • the first buffer time length change information is the buffer collected within the time range of the two video frame intervals. Zone time length change information, the second buffer time length change information bit rate decision interval buffer time length change information.
  • the evaluation value obtaining unit is configured to simultaneously input the network throughput at the third moment, the information about the change in the second time length, and the target decision code rate at the first moment into the second model.
  • the first evaluation value is output.
  • the device further includes:
  • the target code rate decision determining unit at the third moment is configured to simultaneously input the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment into the first model to obtain the third moment The target decision bit rate.
  • the decision evaluation model determining unit is configured to update the model parameters of the second model based on the network throughput at the fourth moment, the third time length change information, and the target decision code rate at the third moment, until any iterative process meets the second
  • the iterative end condition is used to obtain the decision evaluation model, where the decision evaluation model is the second model obtained from the iterative process that satisfies the second iteration end condition, where the fourth moment is the next video data transmission moment of the third moment, and the third
  • the time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the fourth moment.
  • the first probability output unit is further configured to perform, during the first model training process, the sample data is input to the first model, and the first model outputs multiple first models corresponding to multiple second decision code rates.
  • the sample data includes historical decision code rate, historical buffer time length information, historical buffer time length change information, and historical network throughput.
  • the target decision code rate determining unit at the first moment is further configured to determine the sample target code rate, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the time length change information acquisition unit is also configured to execute input of the target code rate of the sample into the simulation interactive environment to obtain sample time length change information of the buffer simulation module in the simulation interactive environment.
  • the evaluation value acquisition unit is further configured to input the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission time into the second model, and the second model outputs the second evaluation value.
  • the model parameter update unit is further configured to perform update of the model parameter of the first model based on the second evaluation value.
  • Figure 9 is a block diagram of a code rate decision device. 9, the device includes a second probability output unit 901, a second target decision code rate determination unit 902, and a code rate adjustment unit 903.
  • the second probability output unit 901 is configured to simultaneously input the network throughput at the fifth moment, the first parameter change information, and the target decision rate at the sixth moment into the rate decision model, and the rate decision model outputs more information.
  • the second target decision code rate determining unit 902 is configured to determine the target decision code rate at the fifth moment, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that the third target condition is met.
  • the code rate adjustment unit 903 is configured to execute the target decision code rate based on the fifth moment to adjust the code rate of the video data, and the code rate decision model is the code rate decision model trained by the aforementioned decision model training device.
  • the device further includes:
  • the rate decision model update unit is configured to execute the target decision rate based on the fifth moment and the network throughput at the seventh moment to update the model parameters of the rate decision model.
  • the seventh moment is the next video data at the fifth moment. Transmission moment.
  • an embodiment of the present disclosure also provides an electronic device.
  • the electronic device includes:
  • processor 1001 is configured to perform the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple first decision code rates corresponding to multiple first decision rates.
  • Probability where the second time is the last bit rate decision time at the first time, and the first time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the first time.
  • the target decision code rate at the first moment is determined, and the target decision code rate at the first moment is a first decision code rate with a first probability of meeting the first target condition.
  • the target decision code rate at the first moment into the simulation interactive environment to obtain the second time length change information, where the second time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the third moment, Among them, the third moment is the next video data transmission moment of the first moment.
  • a first evaluation value is obtained based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time, where the first evaluation value is the evaluation value of the target decision code rate at the first time.
  • the model parameters of the first model are updated until any iteration process meets the first iteration end condition, and the rate decision model is obtained.
  • the rate decision model is the first iteration process that meets the first iteration end condition. One model.
  • the simulation interactive environment further includes an encoder simulation module and a transmission simulation module, and the processor 1001 is configured to perform the following steps:
  • the target decision code rate at the first moment is input to the encoder simulation module, and the encoder simulation module sends the video data of the target decision code rate at the first moment to the buffer simulation module.
  • the video data is extracted from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the second time length change information is acquired based on the difference between the storage capacity of the video data at the first time and the third time by the buffer emulation module.
  • the network throughput includes a first network throughput and a second network throughput, where the first network throughput is the network throughput collected within a time range of two video frame intervals, and the second network throughput The quantity is the network throughput of the bit rate decision interval.
  • the time length change information of the buffer emulation module includes the first buffer time length change information and the second buffer time length change information.
  • the first buffer time length change information is the buffer collected within the time range of the two video frame intervals. Zone time length change information, the second buffer time length change information bit rate decision interval buffer time length change information.
  • the processor 1001 is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the second model, and the second model outputs the first evaluation value.
  • the processor 1001 is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the first model to obtain the target decision code rate at the third moment.
  • the decision evaluation model is the second model obtained by the iterative process that satisfies the end condition of the second iteration, where the fourth moment is the next video data transmission moment at the third moment, and the third time length change information is the simulation at the fourth moment Time length change information of the buffer emulation module in the interactive environment.
  • the processor 1001 is configured to perform the following steps:
  • sample data is input to the first model, and the first model outputs multiple second probabilities corresponding to multiple second decision code rates.
  • the sample data includes historical decision code rate and historical buffer time length Information, historical buffer time length change information, and historical network throughput.
  • the sample target code rate is determined, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the sample target code rate is input into the simulation interactive environment, and the sample time length change information of the buffer simulation module in the simulation interactive environment is obtained.
  • the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission moment are input into the second model, and the second model outputs the second evaluation value.
  • the model parameters of the first model are updated.
  • the embodiments of the present disclosure also provide an electronic device.
  • the electronic device includes:
  • a memory 1102 for storing instructions executable by the processor 1101.
  • processor 1101 is configured to perform the following steps:
  • the network throughput at the fifth moment, the first parameter change information, and the target decision rate at the sixth moment are simultaneously input into the rate decision model, and the rate decision model outputs multiple third decision rates corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is determined, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition.
  • the code rate of the video data is adjusted, and the code rate decision model is a code rate decision model trained by the electronic device of claim 17.
  • the processor 1101 is configured to perform the following steps:
  • the model parameters of the code rate decision model are updated, and the seventh time is the next video data transmission time at the fifth time.
  • the above-mentioned processor is a central processing unit (CPU), or other general-purpose processors, digital signal processing (DSP), or application specific integrated circuit (ASIC). ), field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or any conventional processor. It is worth noting that the processor may be a processor that supports an advanced reduced instruction set machine (advanced RISC machines, ARM) architecture.
  • the aforementioned memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.
  • the memory may also include non-volatile random access memory.
  • the memory can also store device type information.
  • the memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory can be read-only memory (ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), and electrically available Erase programmable read-only memory (electrically, EEPROM) or flash memory.
  • Volatile memory may be random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting illustration, many forms of RAM are available.
  • static random access memory static random access memory
  • dynamic random access memory dynamic random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM synchronous dynamic random access memory
  • double data rate synchronous dynamic random access Memory double data random SDRAM, DDR SDRAM
  • enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM
  • serial link DRAM SLDRAM
  • direct memory bus random access memory direct rambus RAM
  • the present disclosure provides a storage medium.
  • the electronic device can execute the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple first decision code rates corresponding to multiple first decision rates.
  • Probability where the second time is the last bit rate decision time at the first time, and the first time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the first time.
  • the target decision code rate at the first moment is determined, and the target decision code rate at the first moment is a first decision code rate with a first probability of meeting the first target condition.
  • the target decision code rate at the first moment into the simulation interactive environment to obtain the second time length change information, where the second time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the third moment, Among them, the third moment is the next video data transmission moment of the first moment.
  • the first evaluation value is obtained based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time, and the first evaluation value is the evaluation value of the target decision code rate at the first time.
  • the model parameters of the first model are updated until any iteration process meets the first iteration end condition, and the rate decision model is obtained.
  • the rate decision model is the first iteration process that meets the first iteration end condition. One model.
  • the simulation interactive environment further includes an encoder simulation module and a sending simulation module, and the electronic device is configured to perform the following steps:
  • the target decision code rate at the first moment is input to the encoder simulation module, and the encoder simulation module sends the video data of the target decision code rate at the first moment to the buffer simulation module.
  • the video data is extracted from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the second time length change information is obtained.
  • the network throughput includes a first network throughput and a second network throughput, where the first network throughput is the network throughput collected within a time range of two video frame intervals, and the second network throughput The quantity is the network throughput of the bit rate decision interval.
  • the time length change information of the buffer emulation module includes the first buffer time length change information and the second buffer time length change information.
  • the first buffer time length change information is the buffer collected within the time range of the two video frame intervals. Zone time length change information, the second buffer time length change information bit rate decision interval buffer time length change information.
  • the electronic device is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the second model, and the second model outputs the first evaluation value.
  • the electronic device is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the first model to obtain the target decision code rate at the third moment.
  • the decision evaluation model is the second model obtained by the iterative process that satisfies the end condition of the second iteration, where the fourth moment is the next video data transmission moment at the third moment, and the third time length change information is the simulation at the fourth moment Time length change information of the buffer emulation module in the interactive environment.
  • the electronic device is configured to perform the following steps:
  • sample data is input to the first model, and the first model outputs multiple second probabilities corresponding to multiple second decision code rates.
  • the sample data includes historical decision code rate and historical buffer time length Information, historical buffer time length change information, and historical network throughput.
  • the sample target code rate is determined, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the sample target code rate is input into the simulation interactive environment, and the sample time length change information of the buffer simulation module in the simulation interactive environment is obtained.
  • the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission moment are input into the second model, and the second model outputs the second evaluation value.
  • the model parameters of the first model are updated.
  • the present disclosure provides a storage medium.
  • the electronic device can execute the following steps:
  • the network throughput at the fifth time, the first parameter change information, and the target decision code rate at the sixth time are simultaneously input into the rate decision model, and the rate decision model outputs multiple third decision rates corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is determined, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition.
  • the code rate of the video data is adjusted, and the code rate decision model is a code rate decision model trained by the electronic device of claim 17.
  • the electronic device is configured to perform the following steps:
  • the model parameters of the code rate decision model are updated, and the seventh time is the next video data transmission time at the fifth time.
  • the present disclosure provides a computer program product.
  • the computer program When the computer program is executed by a computer, it can cause a processor or an electronic device to perform the following steps:
  • the network throughput at the first moment, the first time length change information, and the target decision code rate at the second moment are simultaneously input into the first model, and the first model outputs multiple first decision code rates corresponding to multiple first decision rates.
  • Probability where the second time is the last bit rate decision time at the first time, and the first time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the first time.
  • the target decision code rate at the first moment is determined, and the target decision code rate at the first moment is a first decision code rate with a first probability of meeting the first target condition.
  • the target decision code rate at the first moment into the simulation interactive environment to obtain the second time length change information, where the second time length change information is the time length change information of the buffer simulation module in the simulation interactive environment at the third moment, Among them, the third moment is the next video data transmission moment of the first moment.
  • the first evaluation value is obtained based on the network throughput at the third time, the second time length change information, and the target decision code rate at the first time, and the first evaluation value is the evaluation value of the target decision code rate at the first time.
  • the model parameters of the first model are updated until any iteration process meets the first iteration end condition, and the rate decision model is obtained.
  • the rate decision model is the first iteration process that meets the first iteration end condition. One model.
  • the simulation interactive environment further includes an encoder simulation module and a transmission simulation module, and the processor or electronic device is configured to perform the following steps:
  • the target decision code rate at the first moment is input to the encoder simulation module, and the encoder simulation module sends the video data of the target decision code rate at the first moment to the buffer simulation module.
  • the video data is extracted from the buffer emulation module based on the rate indicated by the sending emulation module.
  • the second time length change information is acquired based on the difference between the storage capacity of the video data at the first time and the third time by the buffer emulation module.
  • the network throughput includes a first network throughput and a second network throughput, where the first network throughput is the network throughput collected within a time range of two video frame intervals, and the second network throughput The quantity is the network throughput of the bit rate decision interval.
  • the time length change information of the buffer emulation module includes the first buffer time length change information and the second buffer time length change information.
  • the first buffer time length change information is the buffer collected within the time range of the two video frame intervals. Zone time length change information, the second buffer time length change information bit rate decision interval buffer time length change information.
  • the processor or electronic device is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the second model, and the second model outputs the first evaluation value.
  • the processor or electronic device is configured to perform the following steps:
  • the network throughput at the third moment, the second time length change information, and the target decision code rate at the first moment are simultaneously input into the first model to obtain the target decision code rate at the third moment.
  • the decision evaluation model is the second model obtained by the iterative process that satisfies the end condition of the second iteration, where the fourth moment is the next video data transmission moment at the third moment, and the third time length change information is the simulation at the fourth moment Time length change information of the buffer emulation module in the interactive environment.
  • the processor or electronic device is configured to perform the following steps:
  • sample data is input to the first model, and the first model outputs multiple second probabilities corresponding to multiple second decision code rates.
  • the sample data includes historical decision code rate and historical buffer time length Information, historical buffer time length change information, and historical network throughput.
  • the sample target code rate is determined, and the sample target code rate is a second decision code rate with a second probability that meets the second target condition.
  • the sample target code rate is input into the simulation interactive environment, and the sample time length change information of the buffer simulation module in the simulation interactive environment is obtained.
  • the sample target code rate, the sample time length change information, and the network bandwidth at the next video data transmission moment are input into the second model, and the second model outputs the second evaluation value.
  • the model parameters of the first model are updated.
  • the present disclosure provides a computer program product.
  • the computer program When the computer program is executed by a computer, it can cause a processor or an electronic device to perform the following steps:
  • the network throughput at the fifth time, the first parameter change information, and the target decision code rate at the sixth time are simultaneously input into the rate decision model, and the rate decision model outputs multiple third decision rates corresponding to multiple third decision rates.
  • the target decision code rate at the fifth moment is determined, and the target decision code rate at the fifth moment is a third decision code rate with a third probability that meets the third target condition.
  • the code rate of the video data is adjusted, and the code rate decision model is a code rate decision model trained by the electronic device of claim 17.
  • the processor or electronic device is configured to perform the following steps:
  • the model parameters of the code rate decision model are updated, and the seventh time is the next video data transmission time at the fifth time.
  • Figure 12 is an example diagram of bandwidth changes over time. Refer to Figure 12 to verify the network bandwidth with a sine wave as the waveform with large fluctuations.
  • the abscissa in the figure is time (seconds)
  • curve a is the real bandwidth change in Mbps
  • curve b is the change in buffer time size in seconds (s)
  • curve c is the bit rate selected by the model, in Mbps
  • the d curve is the throughput of the actual transmitted video data, in Mbps.
  • the video bit rate control method provided by the present disclosure can make the throughput of the actual transmitted video data follow the change of the real bandwidth well, so that the The throughput is almost equal to the actual unpredictable network bandwidth, while keeping the amount of data stored in the buffer at a relatively low level, which not only guarantees the throughput of live video transmission, but also ensures the real-time nature of the live video.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本公开关于一种码率决策模型训练方法以及电子设备，属于数据处理领域。方法包括：电子设备将网络吞吐量、缓冲区仿真模块的时间长度变化信息以及上一个码率决策时刻的目标决策码率输入到第一模型中，第一模型输出多个第一决策码率对应的第一概率，基于第一概率确定目标决策码率，电子设备基于目标决策码率以及下一个视频数据传输时刻的网络吞吐量和缓冲区仿真模块的时间长度变化信息来确定对于目标决策码率的第一评价值，基于第一评价值，更新第一模型的模型参数。

Description

码率决策模型训练方法以及电子设备
本公开要求于2020年1月16日提交的申请号为202010046898.1、发明名称为“码率决策模型训练方法、装置、电子设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及视频直播领域，尤其涉及一种码率决策模型训练方法以及电子设备。
背景技术
网络波动对于视频直播领域来说有十分巨大的影响，为了能在保持一定的清晰度的同时避免卡顿，电子设备需要根据网络波动情况来调整视频流码率。
视频码率控制一般采用下述方法：调整电子设备所发送的视频数据的码率，以使客户端缓存的视频时长维持在一个给定范围内，例如，将客户端的缓存的直播视频时长维持在10s-20s之间，当缓存的直播视频时长小于10s时，则降低传输的码率，降低视频的清晰度，使得相同的视频数据包中携带时长更长的直播视频；当缓存的直播视频时长大于20s时，则提高传输的码率，提高视频的清晰度，使得相同的视频数据包中携带时短更长的直播视频。
发明内容
本公开提供了一种码率决策模型训练方法以及电子设备，本公开的技术方案如下：
一方面，提供了一种码率决策模型训练方法，应用于电子设备，包括：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
一方面，提供了一种码率决策方法，应用于电子设备，包括：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用权利要求1所述的码率决策模型训练方法训练出的码率决策模型。
一方面，提供了一种码率决策模型训练装置，包括：
第一概率输出单元，被配置为执行将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
第一目标决策码率确定单元，被配置为执行确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
时间长度变化信息获取单元，被配置为执行将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
评价值获取单元，被配置为执行基于所述第三时刻的网络吞吐量、所述第二时间长度 变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
模型参数更新单元，基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
一方面，提供了一种码率决策装置，包括：
第二概率输出单元，被配置为执行将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
第二目标决策码率确定单元，被配置为执行确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
码率调整单元，被配置为执行基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为通过权利要求9所述的码率决策模型训练装置训练出的码率决策模型。
一方面，提供了一种电子设备，包括：
处理器；
用于存储所述处理器可执行指令的存储器；
其中，所述处理器被配置为执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信 息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
一方面，提供了一种电子设备，包括：
处理器；
用于存储所述处理器可执行指令的存储器；
其中，所述处理器被配置为执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用上一方面电子设备训练出的码率决策模型。
一方面，提供了一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第 一目标条件的第一决策码率；
将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
一方面，提供了一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用上一方面所述的存储介质训练出的码率决策模型。
一方面，提供了一种计算机程序产品，当所述计算机程序产品中的指令由电子设备的处理器执行时，使得电子设备能够执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信 息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
一方面，提供了一种计算机程序产品，当所述计算机程序产品中的指令由电子设备的处理器执行时，使得电子设备能够执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用上一方面所述的计算机程序产品训练出的码率决策模型。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理，并不构成对本公开的不当限定。
图1是一种视频数据采集类型的示意图。
图2是一种仿真交互环境的结构示意图。
图3是一种码率决策模型训练方法的流程图。
图4是一种码率决策方法的流程图。
图5是一种码率决策模型训练方法的流程图。
图6是一种Actor网络的结构示意图。
图7是一种Critic网络的结构示意图。
图8是一种码率决策模型训练装置的框图。
图9是一种码率决策装置的框图。
图10是一种电子设备的结构图。
图11是一种电子设备的结构图。
图12是一种带宽随时间变化的示例图。
具体实施方式
为了使本领域普通人员更好地理解本公开的技术方案，下面将结合附图，对本公开实施例中的技术方案进行清楚、完整地描述。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。以下本公开实施例中所描述的实施方式并不代表与本相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
下面是对本公开所涉及的一些名词进行的解释：
码率用于表示一定时长的视频块包含信息多少，相同压缩比下码率较大的视频清晰度更高。
缓冲区用于存储还未发送的视频数据，缓冲区的大小是有限的，在直播领域，希望控制缓冲区内存储的视频数据尽量的小，确保直播的实时性。
网络吞吐量是指在单位时间内传输数据量的多少。
本公开实施例通过训练一个码率决策模型，用以根据上一时刻的网络传输信息来预测下一时刻所需码率，下面对本公开实施例采用的码率决策模型训练方法进行介绍，该码率决策模型训练方法包括数据收集、训练环境创建以及训练三个过程。
在数据收集过程中，在一些实施例中，电子设备获取多个电子设备在数据传输过程中用于表示视频传输环境的相关信息，该相关信息包括但不限于历史网络吞吐量W、缓冲区时间长度信息B、历史码率决策R以及历史缓冲区时间长度变化信息ΔB。上述相关信息所对应的数据采集时间尺度可以不同，举例来说，数据采集时间尺度可以包括长间隔和短间隔，其中，长间隔为在两次码率决策之间的时间间隔，短间隔为相邻两个视频帧之间的 时间间隔，当然，该长间隔和短间隔的时长可以根据实际需要进行设置，本公开实施例对此不做的限定。码率决策是指对当前码率进行调整的方式。例如，数据采集的形式可以如图1所示，电子设备能够同时采集到长间隔和短间隔的网络吞吐量W，分别记为W L和W S；还能够同时采集到长间隔和短间隔的缓冲区时间长度信息B，分别记为B L和B S；历史码率决策R可以仅采集长间隔的数据，记为R L；历史缓冲区时间长度变化信息ΔB可以仅采集短间隔的数据，记为ΔB S。在上述数据收集过程中，电子设备获取的不同时间尺度的数据，对码率决策具有不同的意义，短间隔的信息能应对码率决策中的突发情况，而长间隔上的信息能够使码率决策模型更能抓住数据的全局信息，减少误决策情况的发生。
在训练环境创建过程中，在一些实施例中，基本模型架构采用任一种神经网络，例如，采用深度确定性策略梯度(deep deterministic policy gradient，DDPG)、异步优势动作评价算法(asynchronous advantage actor-critic，A3C)或者策略梯度(policy gradients)等，本公开实施例对此不做限定。
在一些实施例中，考虑到实际模型训练时，若将码率决策模型置于真实环境中训练，实际交互的时间为与真实时间同步，那么经历的环境变化情况十分有限，码率决策模型的训练效率较低。本公开实施例中的码率决策模型训练部分可以采用仿真交互环境与模型进行互动本公开实施例采用仿真交互环境与模型进行互动的方式，对码率决策模型进行训练。基于此，可以创建一个仿真交互环境，来模拟实际的网络吞吐量变化情况。电子设备将收集好的真实网络吞吐量随时间变化的记录输入到仿真交互环境中，由仿真交互环境按照收集好的真实网络吞吐量将视频数据发送出去，并获取当前视频传输环境的相关信息。码率决策模型根据从仿真交互环境中获取的视频传输环境的相关信息，输出相应的码率决策。在此之后，决策评价模型会根据码率决策模型输出的决策码率输出评价值。由于码率决策模型做出的码率决策会影响仿真交互环境，使得仿真交互环境发生一定的变化，最终使得反馈给决策评价模型的相关信息也发生变化，整个训练的过程就是上述交互动作的重复，在一些实施例中，获得的价评价值是有利于码率决策学习码率决策的奖励函数。在一些实施例中，仿真交互环境的结构如图2所示，包括三个模块：编码器仿真模块、缓冲区仿真模块以及发送仿真模块。
其中，编码器仿真模块用于接收码率决策模型输出的码率预测，将对应码率的视频数据发送至缓冲区仿真模块。需要说明的是，视频数据的大小除了会受码率大小的影响之外，也会受到视频数据中每一帧大小的影响，编码器仿真模块能够将视频数据的大小编码在满足码率约束的一定范围内随机波动，同时还需要在编码器仿真模块上设定符合直播视频实 际的码率和视频数据大小。
其中，缓冲区仿真模块用于接收和发送视频数据。缓冲区仿真模块可以基于一个容量有限的队列实现，缓冲区仿真模块每隔一定的帧间隔接收编码器仿真模块发送的视频数据，同时将视频数据发送至发送仿真模块。
其中，发送仿真模块用于接收虚拟网络吞吐量，其中虚拟网络吞吐量是用于仿真网络实际可用带宽的变化情况，在一些实施例中，虚拟网络吞吐量是预先收集好的真实带宽随时间变化的记录。发送仿真模块还用于按照网络吞吐量的限制将缓冲区仿真模块中的视频数据发送出去，从而达到按照网络吞吐量决定的速度消耗缓冲区仿真模块视频数据的目的。
在一些实施例中，编码器仿真模块每次向缓冲区仿真模块发送固定时长的视频数据。码率决策模型输出的决策码率会导致单个视频数据大小的变化，例如一个视频数据的时长为10s，当前视频数据的大小为50KB(kilobyte，千字节)，下一时刻码率决策模型输出的决策码率为0.7，那么电子设备就将视频数据当前码率变为上个时刻的码率的0.7倍，那么相应的，一个视频数据的大小就变成了50×0.7＝35KB。
在一些实施例中，网络吞吐量会随着时间的推移发生波动，基于网络吞吐量从缓冲区仿真模块中取出视频数据，也就表示发送仿真模块不是随意从缓冲区仿真模块中取出视频数据，视频数据的取出速度是由当前的网络吞吐量决定的。例如，当前时刻的网络吞吐量为1000KB/s，也就表示发送仿真模块每秒能够从缓冲区仿真模块中取出1000KB的视频数据，如果单个视频数据的大小为50KB，那么就相当于发送仿真模块每秒能够取出20个视频数据；如果一个视频数据的大小为25KB，那么就相当于发送仿真模块每秒能够取出40个视频数据。缓冲区仿真模块的剩余容量是时刻变化的，若当前网络吞吐量较小，那么发送仿真模块从缓冲区仿真模块中取出视频数据的速度较慢，在这种情况下，由于编码器仿真模块向缓冲区发送的视频数据的数量是一定的，那么缓冲区仿真模块内存储的数据变多，相应的，缓冲区的剩余容量会减小。
在一些实施例中，若缓冲区仿真模块内的视频数据达到其容量上限，则按照“先进先出”的原则将视频数据丢弃。
在训练过程中，本公开实施例提供了一种基于仿真交互环境的模型训练过程，基本构思是电子设备通过第一模型，基于样本数据进行码率决策，由第一模型输出一个决策码率，电子设备将决策码率输入到仿真交互环境中，仿真交互环境基于接收到的决策码率调整视频数据的发送码率。电子设备获取在上述过程中仿真交互环境中缓冲区仿真模块的时 间长度变化信息，将缓冲区仿真模块的时间长度变化信息、决策码率和网络吞吐量输入到第二模型中，由第二模型输出评价值，电子设备基于评价值更新第一模型的模型参数。第一模型、仿真交互环境以及第二模型三者的不断交互，最终得到码率决策模型，该码率决策模型具有基于视频传输环境的相关信息预测得到决策码率的能力。码率决策模型的训练过程参见501-507。
图3是一种码率决策模型训练方法的流程图，应用于电子设备，如图3所示，包括以下步骤：
301、电子设备将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
302、电子设备确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
303、电子设备将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
304、电子设备基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一评价值，第一评价值为第一时刻的目标决策码率的评价值。
305、电子设备基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息包括：
将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。
基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。
基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长度变化信息。
在一些实施例中，网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，第二网络吞吐量为码率决策间隔的网络吞吐量。
缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，第一缓冲区时间长度变化信息为两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，第二缓冲区时间长度变化信息码率决策间隔的缓冲区时间长度变化信息。
在一些实施例中，基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一时刻的目标决策码率的第一评价值包括：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，基于第一评价值，更新第一模型的模型参数之后，方法还包括：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中之前，方法还包括：
在第一次模型训练过程中，将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。
确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。
将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。
将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。
基于第二评价值，更新第一模型的模型参数。
图4是一种码率决策方法的流程图，应用于电子设备，如图4所示，包括以下步骤：
401、电子设备将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由码率决策模型输出多个第三决策码率对应的多个第三概率，其中，第六时刻为第五时刻的上一个码率决策时刻，第一参数变化信息为第五时刻的缓冲区的参数变化信息。
402、电子设备确定第五时刻的目标决策码率，第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率。
在一些实施例中，第三概率符合第三目标条件是指，该第三概率是多个第三概率中最高的。
403、电子设备基于第五时刻的目标决策码率，调整视频数据的码率，码率决策模型为采用上述码率决策模型训练方法训练出的码率决策模型。
在一些实施例中，基于第五时刻的目标决策码率，调整视频数据的码率之后，方法还包括：
基于第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新码率决策模型的模型参数，第七时刻为第五时刻的下一个视频数据传输时刻。
图5是一种码率决策模型训练方法的流程图，如图5所示，方法应用于电子设备中，包括以下步骤：
501、电子设备将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，电子设备将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率组成一个第一参数矩阵，并将第一参数矩阵输入到第一模型中，通过第一模型的至少一个权重矩阵与参数矩阵进行相乘，得到多个第一特征向量，将多个第一特征向量映射为多个第一概率。例如，若第一时刻的网络吞吐量W为500Kbps、第二时刻的目标决策码率R为0.7、第一时间长度变化信息ΔB为3％，电子设备生成一个一维的第一参数矩阵[500，0.7，3] T。电子设备将权重矩阵[0.2，1，0.3]与第一参数矩阵相乘， 得到第一特征向量[10，0.7，0.9] T，通过归一化函数(SoftMax)将其映射为多个第一概率，例如[0.76，0.05，0.07] T，其中，第一特征向量中的数字表示对应的第一决策码率的概率。
需要说明的是，在一些实施例中，第一时刻和第二时刻之间的间隔是提前设置好的，定时直接切换。在一些实施例中，第一时刻和第二时刻之间的间隔是电子设备实时确定的间隔，本公开实施例对此不做限定。在一些实施例中，第一决策码率为码率调整的倍数，或者为一个的码率值。若第一决策码率为码率调整的倍数，那么码率决策模型输出的第一概率对应于不同的码率调整倍数，例如0.7，0.8，0.9，1.0，1.05，1.1以及1.15，每个数字用于表示将当前视频数据的码率调整为上一时刻的倍数，如果在多个第一决策码率中，0.7对应的第一概率最高，那么电子设备通过编码器仿真模块将当前视频数据的码率调整为上一时刻的0.7倍。
502、电子设备确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
在一些实施例中，第一概率符合第一目标条件是指该第一概率为多个第一概率中最高的。
503、电子设备将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，电子设备将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。电子设备基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。电子设备基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长度变化信息。
504、电子设备将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，电子设备将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率组成第二参数矩阵。电子设备将第二参数矩阵输入到第二模型中，通过第二模型的至少一个权重矩阵进行运算，得到多个第二特征向量，将多个第二特征向量映射为第一评价值。
例如，若第三时刻的网络吞吐量W为450Kbps、第一时刻的目标决策码率R为0.5、 第三时刻的缓冲区仿真模块的时间长度变化信息ΔB为2％，电子设备生成一个一维的第二参数矩阵[450，0.5，2] T。电子设备将权重矩阵[0.1，1，0.5]与第二参数矩阵相乘，得到第二特征向量[4.5，0.5，1] T，通过S型生长曲线(Sigmoid)将其映射为第一评价值，例如0.6。
505、电子设备基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，电子设备基于第一评价值，对第一模型的至少一个权重矩阵进行更新，直至第一模型的损失函数的函数值低于目标阈值，或迭代次数达到目标次数时，第一模型训练完成，得到码率决策模型，其中，目标阈值和目标次数可以根据实际情况进行设置，本公开实施例对此不作限定。
506、电子设备将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
其中，电子设备确定第三时刻的目标决策码率的方法与确定第一时刻的目标决策码率的方法属于同一发明构思，在此不再赘述。
507、电子设备基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
需要说明的是，在501-507之前，也即是在第一模型训练过程中，由于还不存在仿真交互环境中各个部分的参数信息，在一些实施例中，电子设备将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，其中，样本数据包括历史决策码率，历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。电子设备确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。电子设备将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。电子设备将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。电子设备基于第二评价值，更新第一模型的模型参数。在一些实施例中，第二概率符合第二目标条件是指，该第二概率是多个第二概率中最高的。
在一些实施例中，样本数据来源于观看视频的用户端或者来自于服务器，本公开实施例对于样本数据的来源不做限定。
在一些实施例中，电子设备能够同时基于不同的网络带宽训练多个码率决策模型，基于训练得到的多个码率决策模型的模型参数，得到主码率决策模型的参数，主码率决策模型也即是在直播过程中进行码率决策的码率决策模型。例如，多个码率决策模型能够分别获取第一参考数量的训练参数，训练参数中至少包括表示视频传输环境的相关信息以及与表示视频传输环境的相关信息对应的评价值。电子设备通过多个码率决策模型，分别将第一参考数量的训练参数发送至主码率决策模型，电子设备基于第一参考数量的训练参数更新主码率决策模型的模型参数。随后，电子设备通过主码率决策模型将更新后的模型参数分别发送给多个码率决策模型。电子设备控制多个码率决策模型分别将模型参数替换为接收到的模型参数，随后继续在不同的仿真交互环境中训练。重复上述步骤，直至电子设备更新主码率决策模型的模型参数的次数达到参考次数时，结束训练。其中，参考次数可以根据实际需要进行设置，本公开实施例对此不做限定。需要说明的是，本公开实施例提供的主码率决策模型除了在达到参考次数的训练之后停止训练，也可以基于模型的损失函数来确定停止训练的时机，本公开实施例对此不做限定。
下面以采用异步更新强化模型(actor-critic)为例说明本公开实施例提供的码率决策模型的训练方法。其中，码率决策模型为Actor(策略)网络、决策评价模型为Critic(价值)网络，Critic网络用于基于在不同网络吞吐量下选择不同码率时所得到的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，来输出评价值，而Actor网络基于Critic网络输出的评价值调整模型参数，Critic网络基于当前时刻的用于表示视频传输环境的相关信息和上一时刻的决策码率，调整模型参数。换句话说，Critic网络对Actor网络输出的决策码率进行评价，Actor网络利用Critic网络的输出的评价值作为训练目标。
在一些实施例中，Actor网络通过如下公式(1)调模型参数，Critic网络通过如下公式(2)调整模型参数：
Figure PCTCN2020129671-appb-000001
Figure PCTCN2020129671-appb-000002
其中，θ a为Actor的网络参数，α a为Actor网络的学习率，π θ(s t,a t)为Actor网络的码率预测，A(s t,a t)是Critic网络输出的评价值；θ c为Critic网络的参数，α c为Critic网络的学习率，V πθ(s t，θ c)是t时刻的网络传输信息s t和当前Critic网络的参数θ c下Critic网络输出的评价值。
在一些实施例中，在进行码率决策时，仅有Actor网络处于激活状态，而在训练状态下Actor网络和Critic网络同时处于激活状态。
在一些实施例中，Critic网络与Actor网络的最后一个输出层是不同的，Critic网络的最后一个输出层为一个没有激活函数的线性输出层；Actor网络的最后一个输出层为一个SoftMax输出层。除了最后一个输出层，Critic网络和Actor网络的其余结构相同。在一些实施例中，Actor网络的结构如图6所示，Critic网络的结构如图7所示。需要说明的是，Critic网络与Actor网络的结构可以基于实际情况进行设计，本公开实施例对此不做限定。
图8是一种码率决策模型训练装置的框图。参照图8，该装置包括第一概率输出单元801，第一目标决策码率确定单元802、时间长度变化信息确定单元803、评价值获取单元804和模型参数更新单元805。
第一概率输出单元801，被配置为执行将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
第一目标决策码率确定单元802，被配置为执行确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
时间长度变化信息获取单元803，被配置为执行将将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
评价值获取单元804，被配置为执行基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一评价值，第一评价值为第一时刻的目标决策码率的评价值。
模型参数更新单元805，被配置为执行基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，时间长度变化信息获取单元包括：
视频数据发送子单元，被配置为执行将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。
视频数据提取子单元，被配置为执行基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。
时间长度变化信息获取子单元，被配置为执行基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长度变化信息。
在一些实施例中，网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，第二网络吞吐量为码率决策间隔的网络吞吐量。
缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，第一缓冲区时间长度变化信息为两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，第二缓冲区时间长度变化信息码率决策间隔的缓冲区时间长度变化信息。
在一些实施例中，评价值获取单元被配置为执行将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，装置还包括：
第三时刻目标码率决策确定单元，被配置为执行将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
决策评价模型确定单元，被配置为执行基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代 过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，第一概率输出单元还被配置为执行在第一次模型训练过程中，将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。
第一时刻的目标决策码率确定单元，还被配置为执行确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。
时间长度变化信息获取单元，还被配置为执行将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。
评价值获取单元，还被配置为执行将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。
模型参数更新单元，还被配置为执行基于第二评价值，更新第一模型的模型参数。
关于上述装置，其中各个模块执行操作的方式已经在有关该方法中进行了详细描述，此处将不做详细阐述说明。
图9是一种码率决策装置的框图。参照图9，该装置包括第二概率输出单元901、第二目标决策码率确定单元902和码率调整单元903。
第二概率输出单元901，被配置为执行将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由码率决策模型输出多个第三决策码率对应的多个第三概率，其中，第六时刻为第五时刻的上一个码率决策时刻，第一参数变化信息为第五时刻的缓冲区的参数变化信息。
第二目标决策码率确定单元902，被配置为执行确定第五时刻的目标决策码率，第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率。
码率调整单元903，被配置为执行基于第五时刻的目标决策码率，调整视频数据的码率，码率决策模型为上述决策模型训练装置训练出的码率决策模型。
在一些实施例中，装置还包括：
码率决策模型更新单元，被配置为执行基于第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新码率决策模型的模型参数，第七时刻为第五时刻的下一个视频数据传输时刻。
关于上述实施例中的装置，其中各个模块执行操作的方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。
基于相同构思，本公开实施例还提供了一种电子设备，如图10所示，所述电子设备包括：
处理器1001；
用于存储所述处理器1001可执行指令的存储器1002；
其中，处理器1001被配置为执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一评价值，第一评价值为第一时刻的目标决策码率的评价值。
基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，处理器1001被配置为执行以下步骤：
将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。
基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。
基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长度变化信息。
在一些实施例中，网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，第二网络吞吐量为码率决策间隔的网络吞吐量。
缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，第一缓冲区时间长度变化信息为两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，第二缓冲区时间长度变化信息码率决策间隔的缓冲区时间长度变化信息。
在一些实施例中，处理器1001被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，处理器1001被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，处理器1001被配置为执行以下步骤：
在第一次模型训练过程中，将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。
确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。
将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。
将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。
基于第二评价值，更新第一模型的模型参数。
基于相同构思，本公开实施例还提供了一种电子设备，如图11所示，电子设备包括：
处理器1101。
用于存储处理器1101可执行指令的存储器1102。
其中，处理器1101被配置为执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由码率决策模型输出多个第三决策码率对应的多个第三概率，其中，第六时刻为第五时刻的上一个码率决策时刻，第一参数变化信息为第五时刻的缓冲区的参数变化信息。
确定第五时刻的目标决策码率，第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率。
基于第五时刻的目标决策码率，调整视频数据的码率，码率决策模型为采用权利要求17的电子设备训练出的码率决策模型。
在一些实施例中，处理器1101被配置为执行以下步骤：
基于第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新码率决策模型的模型参数，第七时刻为第五时刻的下一个视频数据传输时刻。
在一些实施例中，上述处理器是中央处理器(central processing unit，CPU)，或者是其他通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field－programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者是任何常规的处理器等。值得说明的是，处理器可以是支持进阶精简指令集机器(advanced RISC machines，ARM)架构的处理器。
在一些实施例中，上述存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据。存储器还可以包括非易失性随机存取存储器。例如，存储器还可以存储设备类型的信息。
该存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically，EEPROM)或闪存。易失性存储器 可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用。例如，静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic random access memory，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data random SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DRRAM)。
本公开提供了一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一评价值，第一评价值为第一时刻的目标决策码率的评价值。
基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，电子设备被配置为执行以下步骤：
将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。
基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。
基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长 度变化信息。
在一些实施例中，网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，第二网络吞吐量为码率决策间隔的网络吞吐量。
缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，第一缓冲区时间长度变化信息为两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，第二缓冲区时间长度变化信息码率决策间隔的缓冲区时间长度变化信息。
在一些实施例中，电子设备被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，电子设备被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，电子设备被配置为执行以下步骤：
在第一次模型训练过程中，将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。
确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。
将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。
将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。
基于第二评价值，更新第一模型的模型参数。
本公开提供了一种存储介质，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由码率决策模型输出多个第三决策码率对应的多个第三概率，其中，第六时刻为第五时刻的上一个码率决策时刻，第一参数变化信息为第五时刻的缓冲区的参数变化信息。
确定第五时刻的目标决策码率，第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率。
基于第五时刻的目标决策码率，调整视频数据的码率，码率决策模型为采用权利要求17的电子设备训练出的码率决策模型。
在一些实施例中，电子设备被配置为执行以下步骤：
基于第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新码率决策模型的模型参数，第七时刻为第五时刻的下一个视频数据传输时刻。
本公开提供了一种计算机程序产品，当计算机程序被计算机执行时，可以使得处理器或电子设备执行以下步骤：
将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由第一模型输出多个第一决策码率对应的多个第一概率，其中，第二时刻为第一时刻的上一个码率决策时刻，第一时间长度变化信息为第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
确定第一时刻的目标决策码率，第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率。
将第一时刻的目标决策码率输入仿真交互环境，获取第二时间长度变化信息，其中，第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，第三时刻为第一时刻的下一个视频数据传输时刻。
基于第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率，获取第一评价值，第一评价值为第一时刻的目标决策码率的评价值。
基于第一评价值，更新第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，码率决策模型为满足第一迭代结束条件的迭代过程所得到的第一模型。
在一些实施例中，仿真交互环境还包括编码器仿真模块以及发送仿真模块，处理器或电子设备被配置为执行以下步骤：
将第一时刻的目标决策码率输入编码器仿真模块，由编码器仿真模块向缓冲区仿真模块发送第一时刻的目标决策码率的视频数据。
基于发送仿真模块所指示的速率从缓冲区仿真模块中提取视频数据。
基于缓冲区仿真模块在第一时刻和第三时刻存储视频数据的容量差，获取第二时间长度变化信息。
在一些实施例中，网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，第二网络吞吐量为码率决策间隔的网络吞吐量。
缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，第一缓冲区时间长度变化信息为两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，第二缓冲区时间长度变化信息码率决策间隔的缓冲区时间长度变化信息。
在一些实施例中，处理器或电子设备被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第二模型中，由第二模型输出第一评价值。
在一些实施例中，处理器或电子设备被配置为执行以下步骤：
将第三时刻的网络吞吐量、第二时间长度变化信息以及第一时刻的目标决策码率同时输入到第一模型中，得到第三时刻的目标决策码率。
基于第四时刻的网络吞吐量、第三时间长度变化信息和第三时刻的目标决策码率更新第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，决策评价模型为满足第二迭代结束条件的迭代过程所得到的第二模型，其中，第四时刻为第三时刻的下一个视频数据传输时刻，第三时间长度变化信息为第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
在一些实施例中，处理器或电子设备被配置为执行以下步骤：
在第一次模型训练过程中，将样本数据输入第一模型，由第一模型输出多个第二决策码率对应的多个第二概率，样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量。
确定样本目标码率，样本目标码率为第二概率符合第二目标条件的第二决策码率。
将样本目标码率输入到仿真交互环境中，获取仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息。
将样本目标码率、样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由第二模型输出第二评价值。
基于第二评价值，更新第一模型的模型参数。
本公开提供了一种计算机程序产品，当计算机程序被计算机执行时，可以使得处理器或电子设备执行以下步骤：
将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由码率决策模型输出多个第三决策码率对应的多个第三概率，其中，第六时刻为第五时刻的上一个码率决策时刻，第一参数变化信息为第五时刻的缓冲区的参数变化信息。
确定第五时刻的目标决策码率，第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率。
基于第五时刻的目标决策码率，调整视频数据的码率，码率决策模型为采用权利要求17的电子设备训练出的码率决策模型。
在一些实施例中，处理器或电子设备被配置为执行以下步骤：
基于第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新码率决策模型的模型参数，第七时刻为第五时刻的下一个视频数据传输时刻。
图12是一种带宽随时间变化的示例图，参见图12，选取了波动较大的波形为正弦波的网络带宽验证。图中横坐标为时间(秒)，a曲线为真实带宽变化，单位为Mbps，b曲线为缓冲区时间大小的变化单位为秒(s)，c曲线为模型选择的码率，单位为Mbps，d曲线为实际发送视频数据的吞吐量，单位为Mbps，可以看出本公开提供的视频码率控制方法能够使得实际发送视频数据的吞吐量很好地跟随真实带宽的变化，使得发送视频数据的吞吐量几乎和实际不可预知的网络带宽相等，同时保持了缓冲区中存储的数据量在相对较低的水平，这样既保证了视频直播发送的吞吐量，又保证了视频直播的实时性。
本领域技术人员在考虑说明书及实践这里的公开后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应 性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开的真正范围和精神由下面的权利要求指出。
应当理解的是，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。

Claims (26)

  1. 一种码率决策模型训练方法，应用于电子设备，包括：
    将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
    确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
    将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
    基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
    基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
  2. 根据权利要求1所述的码率决策模型训练方法，其中，所述仿真交互环境还包括编码器仿真模块以及发送仿真模块，所述将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息包括：
    将所述第一时刻的目标决策码率输入所述编码器仿真模块，由所述编码器仿真模块向所述缓冲区仿真模块发送所述第一时刻的目标决策码率的视频数据；
    基于所述发送仿真模块所指示的速率从所述缓冲区仿真模块中提取所述视频数据；
    基于所述缓冲区仿真模块在所述第一时刻和所述第三时刻存储所述视频数据的容量差，获取所述第二时间长度变化信息。
  3. 根据权利要求1所述的码率决策模型训练方法，其中，所述网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，所述第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，所述第二网络吞吐量为码率决策间隔的网络吞吐量；
    所述缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，所述第一缓冲区时间长度变化信息为所述两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，所述第二缓冲区时间长度变化信息所述码率决策间隔的缓冲区时间长度变化信息。
  4. 根据权利要求1所述的码率决策模型训练方法，其中，所述基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取所述第一时刻的目标决策码率的第一评价值包括：
    将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到第二模型中，由所述第二模型输出所述第一评价值。
  5. 根据权利要求4所述的码率决策模型训练方法，其中，所述基于所述第一评价值，更新所述第一模型的模型参数之后，所述方法还包括：
    将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到所述第一模型中，得到所述第三时刻的目标决策码率；
    基于第四时刻的网络吞吐量、第三时间长度变化信息和所述第三时刻的目标决策码率，更新所述第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，所述决策评价模型为满足所述第二迭代结束条件的迭代过程所得到的第二模型，其中，所述第四时刻为所述第三时刻的下一个视频数据传输时刻，所述第三时间长度变化信息为所述第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
  6. 根据权利要求1所述的码率决策模型训练方法，其中，所述将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中之前，所述方法还包括：
    在第一次模型训练过程中，将样本数据输入所述第一模型，由所述第一模型输出多个第二决策码率对应的多个第二概率，所述样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量；
    确定样本目标码率，所述样本目标码率为第二概率符合第二目标条件的第二决策码率；
    将所述样本目标码率输入到所述仿真交互环境中，获取所述仿真交互环境中的缓冲区 仿真模块的样本时间长度变化信息；
    将所述样本目标码率、所述样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由所述第二模型输出第二评价值；
    基于所述第二评价值，更新所述第一模型的模型参数。
  7. 一种码率决策方法，应用于电子设备，包括：
    将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
    确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
    基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用权利要求1所述的码率决策模型训练方法训练出的码率决策模型。
  8. 根据权利要求7所述的码率决策方法，其中，所述基于所述第五时刻的目标决策码率，调整视频数据的码率之后，所述方法还包括：
    基于所述第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新所述码率决策模型的模型参数，所述第七时刻为所述第五时刻的下一个视频数据传输时刻。
  9. 一种码率决策模型训练装置，包括：
    第一概率输出单元，被配置为执行将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
    第一目标决策码率确定单元，被配置为执行确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
    时间长度变化信息获取单元，被配置为执行将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时 刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
    评价值获取单元，被配置为执行基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
    模型参数更新单元，基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
  10. 根据权利要求9所述的码率决策模型训练装置，其中，所述仿真交互环境还包括编码器仿真模块以及发送仿真模块，所述时间长度变化信息获取单元包括：
    视频数据发送子单元，被配置为执行将所述第一时刻的目标决策码率输入所述编码器仿真模块，由所述编码器仿真模块向所述缓冲区仿真模块发送所述第一时刻的目标决策码率的视频数据；
    视频数据提取子单元，被配置为执行基于所述发送仿真模块所指示的速率从所述缓冲区仿真模块中提取所述视频数据；
    时间长度变化信息获取子单元，被配置为执行基于所述缓冲区仿真模块在所述第一时刻和所述第三时刻存储所述视频数据的容量差，获取所述第二时间长度变化信息。
  11. 根据权利要求9所述的码率决策模型训练装置，其中，所述网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，所述第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，所述第二网络吞吐量为码率决策间隔的网络吞吐量；
    所述缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，所述第一缓冲区时间长度变化信息为所述两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，所述第二缓冲区时间长度变化信息所述码率决策间隔的缓冲区时间长度变化信息。
  12. 根据权利要求9所述的码率决策模型训练装置，其中，所述评价值获取单元被配置为执行将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到第二模型中，由所述第二模型输出所述第一评价值。
  13. 根据权利要求12所述的码率决策模型训练装置，其中，所述装置还包括：
    第三时刻目标码率决策确定单元，被配置为执行将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到所述第一模型中，得到所述第三时刻的目标决策码率；
    决策评价模型确定单元，被配置为执行基于所述第四时刻的网络吞吐量、所述第三时间长度变化信息和所述第三时刻的目标决策码率更新所述第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，所述决策评价模型为满足所述第二迭代结束条件的迭代过程所得到的第二模型，其中，所述第四时刻为所述第三时刻的下一个视频数据传输时刻，所述第三时间长度变化信息为所述第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
  14. 根据权利要求9所述的码率决策模型训练装置，其中，所述第一概率输出单元还被配置为执行在第一次模型训练过程中，将样本数据输入所述第一模型，由所述第一模型输出多个第二决策码率对应的多个第二概率，所述样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量；
    所述第一目标决策码率确定单元，还被配置为执行确定样本目标码率，所述样本目标码率为第二概率符合第二目标条件的第二决策码率；
    所述时间长度变化信息获取单元，还被配置为执行将所述样本目标码率输入到所述仿真交互环境中，获取所述仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息；
    所述评价值获取单元，还被配置为执行将所述样本目标码率、所述样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由所述第二模型输出第二评价值；
    所述模型参数更新单元，还被配置为执行基于所述第二评价值，更新所述第一模型的模型参数。
  15. 一种码率决策装置，包括：
    第二概率输出单元，被配置为执行将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
    第二目标决策码率确定单元，被配置为执行确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
    码率调整单元，被配置为执行基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为通过权利要求9所述的码率决策模型训练装置训练出的码率决策模型。
  16. 根据权利要求15所述的码率决策装置，其中，所述装置还包括：
    码率决策模型更新单元，被配置为执行基于所述第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新所述码率决策模型的模型参数，所述第七时刻为所述第五时刻的下一个视频数据传输时刻。
  17. 一种电子设备，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行以下步骤：
    将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
    确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
    将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
    基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
    基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
  18. 根据权利要求17所述的电子设备，其中，所述仿真交互环境还包括编码器仿真模块以及发送仿真模块，所述处理器被配置为执行以下步骤：
    将所述第一时刻的目标决策码率输入所述编码器仿真模块，由所述编码器仿真模块向所述缓冲区仿真模块发送所述第一时刻的目标决策码率的视频数据；
    基于所述发送仿真模块所指示的速率从所述缓冲区仿真模块中提取所述视频数据；
    基于所述缓冲区仿真模块在所述第一时刻和所述第三时刻存储所述视频数据的容量差，获取所述第二时间长度变化信息。
  19. 根据权利要求17所述的电子设备，其中，所述网络吞吐量包括第一网络吞吐量以及第二网络吞吐量，其中，所述第一网络吞吐量为两个视频帧间隔的时间范围内采集到的网络吞吐量，所述第二网络吞吐量为码率决策间隔的网络吞吐量；
    所述缓冲区仿真模块的时间长度变化信息包括第一缓冲区时间长度变化信息和第二缓冲区时间长度变化信息，所述第一缓冲区时间长度变化信息为所述两个视频帧间隔的时间范围内采集到的缓冲区时间长度变化信息，所述第二缓冲区时间长度变化信息所述码率决策间隔的缓冲区时间长度变化信息。
  20. 根据权利要求17所述的电子设备，其中，所述处理器被配置为执行以下步骤：
    将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到第二模型中，由所述第二模型输出所述第一评价值。
  21. 根据权利要求20所述的电子设备，其中，所述处理器被配置为执行以下步骤：
    将所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率同时输入到所述第一模型中，得到所述第三时刻的目标决策码率；
    基于所述第四时刻的网络吞吐量、所述第三时间长度变化信息和所述第三时刻的目标决策码率更新所述第二模型的模型参数，直至任一迭代过程满足第二迭代结束条件，得到决策评价模型，其中，所述决策评价模型为满足所述第二迭代结束条件的迭代过程所得到的第二模型，其中，所述第四时刻为所述第三时刻的下一个视频数据传输时刻，所述第三时间长度变化信息为所述第四时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息。
  22. 根据权利要求17所述的电子设备，其中，所述处理器被配置为执行以下步骤：
    在第一次模型训练过程中，将样本数据输入所述第一模型，由所述第一模型输出多个第二决策码率对应的多个第二概率，所述样本数据包括历史决策码率、历史缓冲区时间长度信息、历史缓冲区时间长度变化信息以及历史网络吞吐量；
    确定样本目标码率，所述样本目标码率为第二概率符合第二目标条件的第二决策码率；
    将所述样本目标码率输入到所述仿真交互环境中，获取所述仿真交互环境中的缓冲区仿真模块的样本时间长度变化信息；
    将所述样本目标码率、所述样本时间长度变化信息以及下一个视频数据传输时刻的网络带宽输入到第二模型中，由所述第二模型输出第二评价值；
    基于所述第二评价值，更新所述第一模型的模型参数。
  23. 一种电子设备，包括：
    处理器；
    用于存储所述处理器可执行指令的存储器；
    其中，所述处理器被配置为执行以下步骤：
    将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
    确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
    基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用权利要求17所述的电子设备训练出的码率决策模型。
  24. 根据权利要求23所述的电子设备，其中，所述处理器被配置为执行以下步骤：
    基于所述第五时刻的目标决策码率以及第七时刻的网络吞吐量，更新所述码率决策模型的模型参数，所述第七时刻为所述第五时刻的下一个视频数据传输时刻。
  25. 一种存储介质，其中，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下步骤：
    将第一时刻的网络吞吐量、第一时间长度变化信息以及第二时刻的目标决策码率同时输入到第一模型中，由所述第一模型输出多个第一决策码率对应的多个第一概率，其中，所述第二时刻为所述第一时刻的上一个码率决策时刻，所述第一时间长度变化信息为所述第一时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息；
    确定所述第一时刻的目标决策码率，所述第一时刻的目标决策码率为第一概率符合第一目标条件的第一决策码率；
    将所述第一时刻的目标决策码率输入所述仿真交互环境，获取第二时间长度变化信息，其中，所述第二时间长度变化信息为第三时刻的仿真交互环境中的缓冲区仿真模块的时间长度变化信息，其中，所述第三时刻为所述第一时刻的下一个视频数据传输时刻；
    基于所述第三时刻的网络吞吐量、所述第二时间长度变化信息以及所述第一时刻的目标决策码率，获取第一评价值，所述第一评价值为所述第一时刻的目标决策码率的评价值；
    基于所述第一评价值，更新所述第一模型的模型参数，直至任一迭代过程满足第一迭代结束条件，得到码率决策模型，所述码率决策模型为满足所述第一迭代结束条件的迭代过程所得到的第一模型。
  26. 一种存储介质，其中，当所述存储介质中的指令由电子设备的处理器执行时，使得所述电子设备能够执行以下步骤：
    将第五时刻的网络吞吐量、第一参数变化信息以及第六时刻的目标决策码率同时输入到码率决策模型中，由所述码率决策模型输出多个第三决策码率对应的多个第三概率，其中，所述第六时刻为所述第五时刻的上一个码率决策时刻，所述第一参数变化信息为所述第五时刻的缓冲区的参数变化信息；
    确定所述第五时刻的目标决策码率，所述第五时刻的目标决策码率为第三概率符合第三目标条件的第三决策码率；
    基于所述第五时刻的目标决策码率，调整视频数据的码率，所述码率决策模型为采用权利要求25所述的存储介质训练出的码率决策模型。
PCT/CN2020/129671 2020-01-16 2020-11-18 码率决策模型训练方法以及电子设备 WO2021143344A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20914063.1A EP3968648A4 (en) 2020-01-16 2020-11-18 BIT RATE DECISION MODEL TRAINING METHOD AND ELECTRONIC DEVICE
US17/562,687 US20220124387A1 (en) 2020-01-16 2021-12-27 Method for training bit rate decision model, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010046898.1A CN113132765A (zh) 2020-01-16 2020-01-16 码率决策模型训练方法、装置、电子设备及存储介质
CN202010046898.1 2020-01-16

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/562,687 Continuation US20220124387A1 (en) 2020-01-16 2021-12-27 Method for training bit rate decision model, and electronic device

Publications (1)

Publication Number Publication Date
WO2021143344A1 true WO2021143344A1 (zh) 2021-07-22

Family

ID=76771700

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/129671 WO2021143344A1 (zh) 2020-01-16 2020-11-18 码率决策模型训练方法以及电子设备

Country Status (4)

Country Link
US (1) US20220124387A1 (zh)
EP (1) EP3968648A4 (zh)
CN (1) CN113132765A (zh)
WO (1) WO2021143344A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113259730B (zh) * 2021-07-06 2021-12-14 北京达佳互联信息技术有限公司 用于直播的码率调整方法及装置
CN114039916B (zh) * 2021-10-21 2022-09-16 北京邮电大学 面向实时视频质量优化的深度混合模型流量控制方法、装置和存储介质
CN114827683B (zh) * 2022-04-18 2023-11-07 天津大学 基于强化学习的视频自适应码率控制***及方法
CN118175356A (zh) * 2022-12-09 2024-06-11 中兴通讯股份有限公司 视频传输方法、装置、设备及存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101188752A (zh) * 2007-12-18 2008-05-28 方春 一种基于相关性的自适应码率控制方法
US7400588B2 (en) * 2003-08-01 2008-07-15 Thomson Licensing Dynamic rate adaptation using neural networks for transmitting video data
CN108063961A (zh) * 2017-12-22 2018-05-22 北京联合网视文化传播有限公司 一种基于强化学习的自适应码率视频传输方法以及***
CN109218744A (zh) * 2018-10-17 2019-01-15 华中科技大学 一种基于drl的比特率自适应无人机视频流传输方法
CN109690576A (zh) * 2016-07-18 2019-04-26 渊慧科技有限公司 在多个机器学习任务上训练机器学习模型
CN110149534A (zh) * 2019-06-12 2019-08-20 深圳市大数据研究院 基于决策树的自适应视频流转码方法及装置
WO2019197715A1 (en) * 2018-04-09 2019-10-17 Nokia Technologies Oy An apparatus, a method and a computer program for running a neural network
CN111031387A (zh) * 2019-11-21 2020-04-17 南京大学 一种监控视频发送端视频编码流速控制的方法

Family Cites Families (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6771644B1 (en) * 1999-09-17 2004-08-03 Lucent Technologies Inc. Program insertion in real time IP multicast
US6665705B1 (en) * 1999-10-19 2003-12-16 International Business Machines Corporation Method and apparatus for proxy replication
US7594249B2 (en) * 2001-05-04 2009-09-22 Entropic Communications, Inc. Network interface device and broadband local area network using coaxial cable
US8392952B2 (en) * 2002-05-03 2013-03-05 Time Warner Cable Enterprises Llc Programming content processing and management system and method
US7792984B2 (en) * 2003-01-23 2010-09-07 International Business Machines Corporation Systems and methods for the distribution of bulk data using multicast routing that mitigates network traffic on subnets
US20080037420A1 (en) * 2003-10-08 2008-02-14 Bob Tang Immediate ready implementation of virtually congestion free guaranteed service capable network: external internet nextgentcp (square waveform) TCP friendly san
US7430222B2 (en) * 2004-02-27 2008-09-30 Microsoft Corporation Media stream splicer
US7983160B2 (en) * 2004-09-08 2011-07-19 Sony Corporation Method and apparatus for transmitting a coded video signal
US8135040B2 (en) * 2005-11-30 2012-03-13 Microsoft Corporation Accelerated channel change
US20080098420A1 (en) * 2006-10-19 2008-04-24 Roundbox, Inc. Distribution and display of advertising for devices in a network
US20090025027A1 (en) * 2007-07-20 2009-01-22 Michael Craner Systems & methods for allocating bandwidth in switched digital video systems based on interest
US8813141B2 (en) * 2007-08-08 2014-08-19 At&T Intellectual Properties I, L.P. System and method of providing video content
US8316409B2 (en) * 2007-10-11 2012-11-20 James Strothmann Simultaneous access to media in a media delivery system
US8578432B2 (en) * 2007-12-07 2013-11-05 Cisco Technology, Inc. Policy control over switched delivery networks
US9094140B2 (en) * 2008-04-28 2015-07-28 Time Warner Cable Enterprises Llc Methods and apparatus for audience research in a content-based network
EP2297957B1 (en) * 2008-07-03 2014-09-03 Telefonaktiebolaget LM Ericsson (publ) Fast channel switching in tv broadcast systems
US8014393B1 (en) * 2008-08-05 2011-09-06 Cisco Technology, Inc. Bandwidth optimized rapid channel change in IP-TV network
US9800926B2 (en) * 2008-08-13 2017-10-24 At&T Intellectual Property I, L.P. Peer-to-peer video data sharing
US8112781B2 (en) * 2008-10-07 2012-02-07 General Instrument Corporation Content delivery system having an edge resource manager performing bandwidth reclamation
US8321887B2 (en) * 2008-11-10 2012-11-27 Time Warner Cable Inc. Displaying enhanced advertisements simultaneously across substantially all channels
US8949888B2 (en) * 2008-12-30 2015-02-03 Verizon Patent And Licensing Inc. Systems and methods for efficient messaging and targeted IP multicast advertisement in communication networks
CA2774480C (en) * 2009-09-15 2023-02-28 Comcast Cable Communications, Llc Control plane architecture for multicast cache-fill
US10264029B2 (en) * 2009-10-30 2019-04-16 Time Warner Cable Enterprises Llc Methods and apparatus for packetized content delivery over a content delivery network
US10003851B2 (en) * 2009-11-24 2018-06-19 Imagine Communications Corp. Managed multiplexing of video in an adaptive bit rate environment
US9158769B2 (en) * 2009-12-28 2015-10-13 Adam Dunstan Systems and methods for network content delivery
US9143813B2 (en) * 2010-02-11 2015-09-22 Beaumaris Networks Inc. Multi-video-service bandwidth allocation
JP5569053B2 (ja) * 2010-03-11 2014-08-13 ソニー株式会社 コンテンツ配信装置、コンテンツ配信方法および送信サーバ
GB2490659A (en) * 2011-05-04 2012-11-14 Nds Ltd Fast channel change using channel packs comprising independently decodable frame segments having differing qualities
US9380079B2 (en) * 2011-06-29 2016-06-28 Cable Television Laboratories, Inc. Content multicasting
US8887214B1 (en) * 2011-07-07 2014-11-11 Cisco Technology, Inc. System and method for unified metadata brokering and policy-based content resolution in a video architecture
US9264508B2 (en) * 2011-08-19 2016-02-16 Time Warner Cable Enterprises Llc Apparatus and methods for reduced switching delays in a content distribution network
US9197907B2 (en) * 2011-10-07 2015-11-24 Ericsson Ab Adaptive ads with advertising markers
US9485526B2 (en) * 2012-07-16 2016-11-01 Time Warner Cable Enterprises Llc Multi-stream shared communication channels
US10708335B2 (en) * 2012-11-16 2020-07-07 Time Warner Cable Enterprises Llc Situation-dependent dynamic bit rate encoding and distribution of content
US9066153B2 (en) * 2013-03-15 2015-06-23 Time Warner Cable Enterprises Llc Apparatus and methods for multicast delivery of content in a content delivery network
US9402107B2 (en) * 2013-03-15 2016-07-26 Time Warner Cable Enterprises Llc Apparatus and methods for delivery of multicast and unicast content in a content delivery network
US9628405B2 (en) * 2014-04-07 2017-04-18 Ericsson Ab Merging multicast ABR and unicast ABR with progressive download ABR in a customer premises device within the same video delivery pipe
US9685166B2 (en) * 2014-07-26 2017-06-20 Huawei Technologies Co., Ltd. Classification between time-domain coding and frequency domain coding
CA2975904C (en) * 2015-02-07 2023-02-14 Zhou Wang Method and system for smart adaptive video streaming driven by perceptual quality-of-experience estimations
US9826261B2 (en) * 2015-09-09 2017-11-21 Ericsson Ab Fast channel change in a multicast adaptive bitrate (MABR) streaming network using multicast repeat segment bursts in a dedicated bandwidth pipe
US9826262B2 (en) * 2015-09-09 2017-11-21 Ericsson Ab Fast channel change in a multicast adaptive bitrate (MABR) streaming network using multicast repeat segment bursts in a shared progressive ABR download pipe
US9788053B2 (en) * 2015-09-09 2017-10-10 Ericsson Ab Fast channel change in a multicast adaptive bitrate (MABR) streaming network using HTTP download segment recovery in a dedicated bandwidth pipe
US10104413B2 (en) * 2016-07-07 2018-10-16 Telefonaktiebolaget Lm Ericsson (Publ) Bandwidth and ABR video QoE management based on OTT video providers and devices
US9888278B2 (en) * 2016-07-07 2018-02-06 Telefonaktiebolaget Lm Ericsson (Publ) Bandwidth and ABR video QoE management based on OTT video providers and devices
US10063902B2 (en) * 2016-12-22 2018-08-28 Cisco Technology, Inc. ABR network profile selection engine
WO2019002465A1 (en) * 2017-06-28 2019-01-03 Deepmind Technologies Limited NEURONAL LEARNING ACTION SELECTION NETWORKS USING APPRENTICESHIP
CN110312143B (zh) * 2019-07-25 2020-10-16 北京达佳互联信息技术有限公司 视频码率控制方法、装置、电子设备及存储介质
US11316794B1 (en) * 2020-01-26 2022-04-26 Zodiac Systems, Llc Method and system for improving adaptive bit rate content and data delivery

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7400588B2 (en) * 2003-08-01 2008-07-15 Thomson Licensing Dynamic rate adaptation using neural networks for transmitting video data
CN101188752A (zh) * 2007-12-18 2008-05-28 方春 一种基于相关性的自适应码率控制方法
CN109690576A (zh) * 2016-07-18 2019-04-26 渊慧科技有限公司 在多个机器学习任务上训练机器学习模型
CN108063961A (zh) * 2017-12-22 2018-05-22 北京联合网视文化传播有限公司 一种基于强化学习的自适应码率视频传输方法以及***
WO2019197715A1 (en) * 2018-04-09 2019-10-17 Nokia Technologies Oy An apparatus, a method and a computer program for running a neural network
CN109218744A (zh) * 2018-10-17 2019-01-15 华中科技大学 一种基于drl的比特率自适应无人机视频流传输方法
CN110149534A (zh) * 2019-06-12 2019-08-20 深圳市大数据研究院 基于决策树的自适应视频流转码方法及装置
CN111031387A (zh) * 2019-11-21 2020-04-17 南京大学 一种监控视频发送端视频编码流速控制的方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3968648A4 *

Also Published As

Publication number Publication date
US20220124387A1 (en) 2022-04-21
EP3968648A4 (en) 2022-08-24
CN113132765A (zh) 2021-07-16
EP3968648A1 (en) 2022-03-16

Similar Documents

Publication Publication Date Title
WO2021143344A1 (zh) 码率决策模型训练方法以及电子设备
US10853726B2 (en) Neural architecture search for dense image prediction tasks
US11694109B2 (en) Data processing apparatus for accessing shared memory in processing structured data for modifying a parameter vector data structure
WO2021259090A1 (zh) 联邦学习的方法、装置和芯片
US11847541B2 (en) Training neural networks using data augmentation policies
WO2021026944A1 (zh) 基于粒子群和神经网络的工业无线流媒体自适应传输方法
CN110413838B (zh) 一种无监督视频摘要模型及其建立方法
WO2021129086A1 (zh) 流量预测方法、装置以及存储介质
CN113034380A (zh) 一种基于改进可变形卷积校正的视频时空超分辨率方法和装置
KR102300903B1 (ko) 데이터 증강 방법 및 장치, 그리고 컴퓨터 프로그램
CN109784153A (zh) 情绪识别方法、装置、计算机设备及存储介质
CN111357018B (zh) 使用神经网络的图像分割
US20220230065A1 (en) Semi-supervised training of machine learning models using label guessing
CN110264407A (zh) 图像超分辨模型训练及重建方法、装置、设备及存储介质
CN113259657A (zh) 基于视频质量分数的dppo码率自适应控制***和方法
CN109558543A (zh) 一种样本采样方法、样本采样装置、服务器和存储介质
CN114863219A (zh) 训练大规模视觉Transformer神经网络
Zhang et al. MIPD: An adaptive gradient sparsification framework for distributed DNNs training
CN114782742A (zh) 基于教师模型分类层权重的输出正则化方法
CN112948101B (zh) 一种面向快速联邦学习的5g边缘设备调度方法
CN108573275B (zh) 一种在线分类微服务的构建方法
EP4042334A1 (en) Depth-parallel training of neural networks
CN112396069B (zh) 基于联合学习的语义边缘检测方法、装置、***及介质
US20230177381A1 (en) Accelerating the Training of Machine Learning (ML) Models via Data Instance Compression
CN118043823A (zh) 逐步展开降噪神经网络

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20914063

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020914063

Country of ref document: EP

Effective date: 20211210

NENP Non-entry into the national phase

Ref country code: DE