CN113573101B - Video coding method, device, equipment and storage medium - Google Patents

Video coding method, device, equipment and storage medium Download PDF

Info

Publication number
CN113573101B
CN113573101B CN202110779065.0A CN202110779065A CN113573101B CN 113573101 B CN113573101 B CN 113573101B CN 202110779065 A CN202110779065 A CN 202110779065A CN 113573101 B CN113573101 B CN 113573101B
Authority
CN
China
Prior art keywords
code rate
target
resolution
real
current video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110779065.0A
Other languages
Chinese (zh)
Other versions
CN113573101A (en
Inventor
张凯明
要瑞宵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Pte Ltd
Original Assignee
Bigo Technology Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bigo Technology Pte Ltd filed Critical Bigo Technology Pte Ltd
Priority to CN202110779065.0A priority Critical patent/CN113573101B/en
Publication of CN113573101A publication Critical patent/CN113573101A/en
Priority to PCT/CN2022/100805 priority patent/WO2023279978A1/en
Application granted granted Critical
Publication of CN113573101B publication Critical patent/CN113573101B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application discloses a video coding method, a video coding device, video coding equipment and a video coding storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring code rate change information corresponding to the current network condition, wherein the code rate change information is used for indicating fluctuation of the current network condition relative to the historical network condition; acquiring parameter change information corresponding to the current video content, wherein the parameter change information is used for indicating the content complexity of the current video content; determining a target resolution based on the code rate variation information and the parameter variation information; the current video frame is encoded with the target resolution. According to the application, the fluctuation of the network quality and the complexity of the video content are considered at the same time when the resolution is regulated, when the fluctuation of the network quality is too frequent, the frequent switching of the resolution is effectively avoided, the respiratory effect generated by the frequent fluctuation of the network quality is reduced, and the video watching and conversation experience of a user is improved.

Description

Video coding method, device, equipment and storage medium
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a video coding method, a video coding device, video coding equipment and a storage medium.
Background
With the development of the mobile internet, the instant messaging technology has penetrated into aspects of daily life and work of people. The combination of the instant messaging technology and the video technology is more birth to more diversified instant messaging scenes such as video call, remote conference, online live broadcast, network education and the like.
Generally, such an instant messaging scenario combined with video is extremely dependent on network quality. During video coding, an encoder must regulate and control a video coding strategy in real time based on network quality so that the consumed code rate of video coding accords with network bandwidth, and thus the problems of playing clamping, playing failure and the like caused by poor network quality can be avoided. Currently, the regulation of video coding strategies is mainly divided into two categories: encoder internal code rate regulation and encoder external frame rate/resolution regulation. The regulation and control of the code rate in the encoder means that the code rate in a certain range is regulated under the scene of fixed frame rate/resolution; when the fluctuation of the network quality is large, the external frame rate/resolution regulation of the encoder is usually carried out so as to stabilize the consumption code rate of video coding.
Compared with frame rate regulation, the method has the advantages that resolution regulation is milder, and application scenes are wider. However, when the network quality fluctuates too frequently, the resolution is switched too frequently, so that obvious breathing effects can be generated, and the video watching and conversation experience of the user are affected.
Disclosure of Invention
The embodiment of the application provides a video coding method, a video coding device, video coding equipment and a video coding storage medium, which can be used for avoiding frequent switching of resolution, and are beneficial to reducing respiratory effects generated by frequent fluctuation of network quality and improving video watching and conversation experience of users. The technical scheme is as follows:
in one aspect, an embodiment of the present application provides a video encoding method, including:
acquiring code rate change information corresponding to a current network condition, wherein the code rate change information is used for indicating fluctuation of the current network condition relative to a historical network condition;
acquiring parameter change information corresponding to current video content, wherein the parameter change information is used for indicating the content complexity of the current video content;
determining a target resolution based on the code rate variation information and the parameter variation information;
and encoding the current video frame by adopting the target resolution.
In another aspect, an embodiment of the present application provides a video encoding apparatus, including:
the code rate information acquisition module is used for acquiring code rate change information corresponding to the current network condition, wherein the code rate change information is used for indicating fluctuation of the current network condition relative to the historical network condition;
The parameter information acquisition module is used for acquiring parameter change information corresponding to the current video content, wherein the parameter change information is used for indicating the content complexity of the current video content;
the resolution determining module is used for determining target resolution based on the code rate change information and the parameter change information;
and the video coding module is used for coding the current video frame by adopting the target resolution.
In yet another aspect, an embodiment of the present application provides a computer device, including a processor and a memory, where the memory stores a computer program, and the computer program is loaded and executed by the processor to implement a video encoding method as described above.
In yet another aspect, embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a video encoding method as described above.
In yet another aspect, embodiments of the present application provide a computer program product which, when run on a computer device, causes the computer device to perform a video encoding method as described above.
The technical scheme provided by the embodiment of the application can bring the following beneficial effects:
The resolution adopted in the process of encoding the current video frame is determined by combining the fluctuation of network quality and the complexity of video content in the video encoding process, so that the resolution can be regulated and controlled more accurately. Because human eyes are more sensitive to picture quality change in a simple scene, the application combines the complexity of video content when regulating and controlling the resolution, considers the sensitivity of human eyes to different content complexity of the video content, and thus, the regulation and control of the resolution is more accurate and comprehensive. In addition, the application considers the fluctuation of the network quality and the complexity of the video content simultaneously when the resolution is regulated and controlled, effectively avoids frequent switching of the resolution when the network quality fluctuation is too frequent, is beneficial to reducing the respiratory effect generated by frequent fluctuation of the network quality and improves the video watching and conversation experience of users.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of an instant messaging scenario according to an embodiment of the present application;
fig. 2 is a flowchart of a video encoding method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of critical block matching provided by one embodiment of the present application;
fig. 4 is a flowchart of a video encoding method according to another embodiment of the present application;
fig. 5 is a block diagram of a video encoding apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of a video encoding apparatus according to another embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.
As can be seen from the description of the background art, the intra-encoder rate regulation is applicable to a scene with a fixed frame rate/resolution, and when the network quality fluctuates greatly, the intra-encoder rate/resolution regulation is generally performed. Next, description will be made on frame rate regulation and resolution regulation, respectively.
Frame rate regulation: the client automatically maintains a frame-level code rate statistics queue for caching the recent actual consumption code rate of the coded frame, further calculates the frame-level average consumption code rate of the coded frame, and before the current frame enters the encoder, judges whether the current frame still has sufficient code rate for coding by comparing the frame-level average consumption code rate of the coded frame with a target frame-level code rate obtained by utilizing actual bandwidth estimation, and if the residual code rate is insufficient, the current frame is directly discarded, and the arrival of the next acquisition frame is waited.
In one example, the frame-level average consumed code rate for the encoded frame is calculated as follows:
whether the current frame needs to be discarded is determined according to the following calculation formula:
wherein R is av Average consumed code rate at frame level for encoded frames, R k The actual consumption code rate of the kth frame is given, and n is the sequence number of the current frame; targteBits is a target frame-level code rate; FPS is the set frame rate; alpha is a threshold coefficient; if the discard flag bit is 1, it indicates that the current frame needs to be discarded, and if the discard flag bit is 0, it indicates that the current frame does not need to be discarded. Based on the above formula, when the ratio between the frame-level average consumption rate of the encoded frame and the target frame-level rate is lower than a certain threshold, the current frame needs to be discarded to ensure the stability of the rate.
Resolution regulation: i.e. a code table scheme. Aiming at videos with different resolutions, various code rates are set in sequence from small to large for encoding and decoding, a reconstructed video is obtained, PSNR (Peak Signal to Noise Ratio ) under various code rates is calculated, and the larger the PSNR value is, the better the video quality is represented. Through a large amount of experimental data, a code rate range with an optimal resolution can be obtained, for example, 270P (standard definition) video has the best video quality at a code rate of 320kbps to 500 kbps. And integrating code rate ranges with proper resolutions to form a code table scheme.
In one example, the code table scheme is as follows:
wherein the levels represent different resolution levels, such as 180P (fluency),270P (standard definition), 360P (high definition); level (level) mr Refers to a default resolution level, such as a default resolution of 270P; upper level and domnlevel represent levels, respectively mr And when the target frame level code rate is out of range, the resolution is required to be switched according to the code table scheme. Illustratively, assume the level above mr 270P, level mr 1 is 180P and domnlevel is 350kps, then the resolution needs to be switched from 270P to 180P when TargteBits fluctuates from 400kps to 300 kbps.
From the above description, the frame rate regulation and control is more focused on guaranteeing the video quality, and the smoothness of the code rate is obtained mainly at the expense of the smoothness of the video, but such regulation and control can cause the video to generate obvious clamping, even to be blocked when serious, and is very unfriendly to the subjective experience of users. In other words, the frame rate is controlled with too great a degree of control, which is suitable for extreme cases. Resolution regulation and control is more focused on guaranteeing the fluency of the video, and the resolution of the video is regulated in real time based on network quality, so that the effect of balancing the video code rate is achieved. Compared with frame rate regulation, the resolution regulation method is relatively mild, is more friendly to subjective experience of users, and is relatively wide in application scene.
However, with resolution regulation, when the network quality fluctuates too frequently, the resolution is switched too frequently, so that a significant respiratory effect occurs. In addition, human eyes can be more sensitive to the picture quality change under simple scenes, at the moment, the resolution regulation and control needs to be careful, and the phenomenon that the picture quality of a video is suddenly reduced or suddenly increased due to overlarge resolution regulation and control force, so that watching and conversation experience of a user are influenced is avoided.
Based on the above, the embodiment of the application provides a video coding method, which can more efficiently and flexibly determine the resolution adopted when the current video frame is coded in the video coding process. The technical scheme provided by the application will be described below by combining several embodiments.
It should be noted that, the "instant messaging" in the following embodiments refers to instant messaging combined with video, such as video call, teleconference, online live broadcast, network education, and so on. The embodiment of the application does not limit the development mode of instant messaging, and a user can develop instant messaging through application programs, web pages, applets and the like with the instant messaging function. Optionally, the application program with the instant messaging function includes an instant messaging application program, a video playing application program, a live broadcast application program, a game application program, an education application program, and the embodiment of the present application is not limited thereto.
It should be further noted that, in the embodiment of the present application, only the processes of video encoding and video decoding are described for convenience, but the present application is not limited to the technical scheme. In practical application, the video coding method provided by the embodiment of the application can also be applied to non-instant communication scenes, for example, the user shares the shot video into scenes such as personal social platforms, webpages and the like, and the video coding method belongs to the protection scope of the application.
Referring to fig. 1, a schematic diagram of an instant messaging scenario according to an embodiment of the application is shown. As shown in fig. 1, the instant messaging scenario includes: a video encoding end 110, a video decoding end 120.
The video encoding end 110 is a generating end of video content in an instant messaging scene, and is configured to generate video content, encode the generated video content, send the encoded video content to the video decoding end 120, and so on. The video decoding end 120 refers to a receiving end of video content in an instant messaging scene, and is configured to decode the received video content, play the decoded video content, and the like. Optionally, the video encoding end 110 and the video decoding end 120 communicate with each other through a network. In one example, the video encoder 110 and the video decoder 120 may communicate directly, or may communicate indirectly through a server (not shown in fig. 1), for example, the video encoder 110 may send the encoded video content to the server, and the server may send the encoded video content to the video decoder 120.
The device type of the video encoding end 110 and/or the video decoding end 120 according to the embodiments of the present application is not limited, and in one example, the video encoding end 110 and/or the video decoding end 120 is any one of the following devices: computer devices, terminal devices, servers, wearable devices, bluetooth devices, vehicle-mounted devices, and the like. In one example, video encoding side 110 and video decoding side 120 are of the same device type, or are of different device types. Of course, in an actual instant messaging scenario, the video encoding end 110 and the video decoding end 120 may be implemented as the same device, that is, the device has the function of the video encoding end 110 and the function of the video decoding end 120. Fig. 1 illustrates only the video encoding end 110 and the video decoding end 120 as different devices, but this does not limit the technical solution of the present application.
It should be noted that, in the embodiment of the present application, the video encoding end 110 and the video decoding end 120 are distinguished for the convenience of describing the video encoding and decoding processes, but this does not limit the technical solution of the present application. In an actual instant messaging scene, the video encoding end 110 can encode and send out video content, and can also receive and decode the video content; the video decoding end 120 may either receive and decode the video content or encode and transmit the video content. In other words, in an actual instant messaging scenario, each party in instant messaging may perform the steps of generating and encoding video content, decoding and playing video content generated by another party, and the like.
Referring to fig. 2, a flowchart of a video encoding method according to an embodiment of the application is shown. The method can be applied to the video coding end 110 of the instant messaging scene shown in fig. 1. As shown in FIG. 2, the method includes the following steps (210-240).
Step 210, obtaining code rate variation information corresponding to the current network condition, where the code rate variation information is used to indicate fluctuation of the current network condition relative to the historical network condition.
The network quality has important influence on the code rate, the resolution and the like adopted in the video coding process, and in order to realize better video coding effect, ensure the definition, the fluency and the like of video playing, the influence of the network quality on the parameters adopted in the video coding needs to be considered. From the above, when the network quality fluctuates greatly, the frame rate and/or resolution needs to be adjusted when the range of code rate adjustment is exceeded. In the embodiment of the application, as the frame rate regulation and control force is too large, the method is only suitable for extreme conditions, and therefore, a milder resolution regulation and control mode is adopted under the condition of large network quality fluctuation.
Therefore, in order to determine whether resolution regulation is required, or to determine the target resolution to be adopted for the current video frame during video encoding, it is necessary to determine the network quality fluctuation condition, i.e. the fluctuation of the current network condition relative to the historical network condition. Since network quality is typically reflected by network bandwidth and code rate is typically associated with network bandwidth, fluctuations in current network conditions relative to historical network conditions are indicated by code rate change information in embodiments of the present application. The historical network condition may be the network quality from a certain time before the current time to the current time, or the network quality from the starting time of encoding the video content to the current time. For other descriptions of the code rate change information, the current network condition, the historical network condition, etc., please refer to the following embodiments, which are not repeated here.
Step 220, obtaining parameter variation information corresponding to the current video content, where the parameter variation information is used to indicate the content complexity of the current video content.
Since human eyes are more sensitive to the picture quality change in a simple scene, under the condition of simple video content, the resolution regulation and control needs to be more careful, and the phenomenon that the picture quality of the video is sharply reduced or sharply increased due to overlarge resolution regulation and control force is avoided. Compared with the prior art, only the influence of network quality on video coding is considered, the video coding method provided by the embodiment of the application also considers the sensitivity of human eyes to different content complexity degrees of video content, thereby avoiding abrupt change of the picture quality of the video.
Therefore, in order to determine the target resolution to be employed for the current video frame during video encoding, it is also necessary to determine the content complexity of the current video content. Generally, when the code rate is stable, the higher the texture complexity of the video frame is, the larger the actual quantization parameters (Quantization Parameter, QP) after encoding are, and the positive correlation relationship is presented between the two parameters, so that the content complexity of the current video content is indicated by the parameter change information of the quantization parameters in the embodiment of the application. For other descriptions of parameter variation information, content complexity, etc., please refer to the following embodiments, which are not repeated here.
In step 230, a target resolution is determined based on the code rate variation information and the parameter variation information.
After the code rate variation information and the parameter variation information are acquired, the target resolution adopted when the current video frame is encoded is determined based on the two aspects of information. In one example, the video encoding end determines that resolution regulation is not needed based on the information of the two aspects, and directly takes the original resolution of the current video frame as the target resolution. In another example, if the video encoding end determines that resolution adjustment is required based on the information of the two aspects, the original resolution of the current video frame is increased or decreased to obtain the target resolution. For other descriptions of the determination of the target resolution, please refer to the following examples, which are not repeated here.
The current video frame is encoded using the target resolution, step 240.
After the video coding end determines the target resolution, the target resolution can be adopted to code the current video frame, so that the video code rate is matched with the network bandwidth, and the problems of video playing blocking or playing failure and the like are avoided. Optionally, the video encoding end encodes the current video frame with the target resolution includes: the video encoding end performs scaling processing on the current video frame based on the target resolution.
Illustratively, where the target resolution is greater than the original resolution of the current video frame, encoding the current video frame with the target resolution includes: up-sampling the current video frame; and under the condition that the resolution of the current video frame after the up-sampling processing is matched with the target resolution, the current video frame after the up-sampling processing is encoded by adopting the target resolution.
Illustratively, where the target resolution is less than the original resolution of the current video frame, encoding the current video frame with the target resolution includes: downsampling the current video frame; and under the condition that the resolution of the current video frame after the downsampling process is matched with the target resolution, the current video frame after the downsampling process is encoded by adopting the target resolution.
Optionally, matching the resolution of the upsampled current video frame (or the downsampled current video frame) to the target resolution includes: the resolution of the current video frame after the up-sampling process (or the current video frame after the down-sampling process) is equal to the target resolution, or the difference between the resolution of the current video frame after the up-sampling process (or the current video frame after the down-sampling process) and the target resolution is smaller than a set threshold.
In summary, according to the technical scheme provided by the embodiment of the application, the resolution adopted in the current video frame coding process is determined by combining the fluctuation of the network quality and the complexity of the video content in the video coding process, so that the resolution can be regulated and controlled more accurately. Because human eyes are more sensitive to picture quality change in a simple scene, the application combines the complexity of video content when regulating and controlling the resolution, considers the sensitivity of human eyes to different content complexity of the video content, and thus, the regulation and control of the resolution is more accurate and comprehensive. In addition, the application considers the fluctuation of the network quality and the complexity of the video content simultaneously when the resolution is regulated and controlled, effectively avoids frequent switching of the resolution when the network quality fluctuation is too frequent, is beneficial to reducing the respiratory effect generated by frequent fluctuation of the network quality and improves the video watching and conversation experience of users.
In the following, an exemplary embodiment of the determination process of the code rate variation information, the parameter variation information, and the target resolution will be described.
In one example, the above step 210 includes the following steps (212-216).
Step 212, determining a real-time target code rate, wherein the real-time target code rate is used for indicating the current network condition.
In order to enable the video frames encoded by the video encoding end to be successfully transmitted under the current network condition, the current network condition needs to be estimated to obtain the encoding code rate for the video encoding end to refer to, namely, the real-time target code rate for indicating the current network condition. Optionally, the video encoding end determines the real-time target code rate by using a bandwidth estimation method, where the bandwidth estimation method includes, but is not limited to: a link delay method, a packet detection bandwidth estimation method and the like.
Step 214, obtaining a historical average code rate, where the historical average code rate is used to indicate historical network conditions.
In order to obtain the fluctuation of the current network condition relative to the historical network condition, the video encoding end also needs to determine the historical average code rate for indicating the historical network condition. Optionally, the historical average code rate is an average code rate from a starting time of video coding to a current time, for example, in the case that the technical scheme of the application is applied to a video call scene, the historical average code rate may be an average code rate from a time of video call opening to the current time; alternatively, the historical average code rate is an average code rate over a period of time prior to the current time, e.g., the historical average code rate is an average code rate over 5 seconds prior to the current time. Based on the above, the video coding end needs to maintain a code rate buffer, and the average value in the code rate buffer is the historical average code rate, so as to be used for indicating the historical network condition.
And step 216, determining code rate change information based on the real-time target code rate and the historical average code rate.
After the video coding end determines the real-time target code rate and the historical average code rate respectively, the real-time target code rate and the historical average code rate can be compared to determine the code rate change information. In one example, the step 216 includes: determining a code rate difference between the real-time target code rate and the historical average code rate; code rate variation information is determined based on the code rate difference. Optionally, the determining the code rate variation information based on the code rate difference value includes: under the condition that the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a positive number, determining code rate change information comprises severe change of the code rate and sudden increase of the code rate; under the condition that the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a negative number, determining code rate change information comprises severe change of the code rate and sudden drop of the code rate; and under the condition that the absolute value of the code rate difference value is smaller than a first threshold value, determining the code rate change information comprises that the code rate is not changed drastically. Optionally, the first threshold is a positive number.
Illustratively, the real-time target code rate is R, the historical average code rate is R av ，R av =avg (buffer_r), where buffer_r is a rate buffer maintained by the video coding end, and avg (·) represents an averaging function; the code rate difference between the real-time target code rate and the historical average code rate is R-R av . Assuming that the first threshold is TH1, the code rate variation information has the following cases:
at |R-R av Under the condition that the I is less than Th1, the code rate change information comprises that the code rate is not changed drastically;
at R-R av Under the condition of more than Th1, the code rate change information comprises that the code rate is changed drastically and the code rate is increased suddenly;
at R av In case R > Th1, the code rate variation information includes a drastic change in code rate and a sudden decrease in code rate.
It should be noted that, for |R-R av The content of the code rate change information in the case of |=th1 is not limited to this, and in practical application, it may be set that the code rate change information in the case includes that the code rate does not change drastically, or that the code rate change information in the case changes drastically, and according to R-R av Is set to the positive and negative of the code rate or the code rate is suddenly reduced. It should be understood that these are all intended to be within the scope of the present application.
In one example, the above step 220 includes the following steps (222-226).
In step 222, a first quantization parameter value is obtained, where the first quantization parameter value is a quantization parameter value of a video frame preceding the current video frame.
In general, in the case where the current video content does not undergo a scene change with respect to the history video content, the difference between the previous and subsequent frames of the video is small, and the content complexity of the video content may be considered to be consistent, and thus, the content complexity of the previous video frame of the current video frame may be used to represent the content complexity of the current video frame. Therefore, in the embodiment of the present application, the video encoding end needs to acquire the quantization parameter value of the previous video frame of the current video frame, i.e. the first quantization parameter value.
Step 224, a second quantization parameter value is obtained, the second quantization parameter value being a mean of the quantization parameter values of the at least one historical video frame.
In order to obtain the content complexity of the current video content, the video encoding end also needs to determine a mean value of quantization parameter values of at least one historical video frame, namely, a second quantization parameter value. Optionally, the at least one historical video frame is all video frames from a starting time of video coding to a current time, for example, in the case that the technical scheme of the application is applied to a video call scene, the at least one historical video frame may be all video frames from a time of video call opening to the current time; alternatively, the at least one historical video frame is all video frames within a period of time prior to the current time; alternatively, the at least one historical video frame is N video frames preceding the current video frame, N being a positive number, optionally N being 5, 10, 15, etc. Based on this, the video encoding end needs to maintain a quantization parameter value buffer, and the average value in the quantization parameter value buffer is the second quantization parameter value, so as to be used for indicating the content complexity of the recent video content. Optionally, the video encoding end maintains a quantization parameter value buffer of length N, N being a positive number, e.g., N being 10.
In step 226, parameter variation information is determined based on the first quantization parameter value and the second quantization parameter value.
The video encoding end compares the first quantization parameter value and the second quantization parameter value after determining the first quantization parameter value and the second quantization parameter value respectively to determine parameter variation information. In one example, the step 226 includes: determining a parameter difference between the first quantization parameter value and the second quantization parameter value; parameter variation information is determined based on the parameter difference value. Optionally, the determining parameter variation information based on the parameter difference value includes: under the condition that the absolute value of the parameter difference value is larger than a second threshold value, determining that the parameter change information indicates that the video content is complex; in case the absolute value of the parameter difference is smaller than the second threshold, it is determined that the parameter change information indicates that the video content is simple. Optionally, the second threshold is a positive number.
Illustratively, the first quantization parameter value is QP and the second quantization parameter value is QP av Wherein QP is av =avg (buffer_qp), where buffer_op is a quantization parameter value buffer maintained by the video coding end, and avg (·) represents an averaging function; the parameter difference between the first quantization parameter value and the second quantization parameter value is QP-QP av . Assuming that the second threshold is Th2, the parameter variation information has the following cases:
at |QP-QP av In case of < Th2, the parameter change information indicates that the video content is simple;
at |QP-QP av In the case of > Th2, the parameter change information indicates that the video content is complex.
One point to be noted is that for |QP-QP av In the case of |=th2, the content indicated by the parameter change information is not limited to this, and in practical application, the parameter change information may be set to indicate that the video content is simple, or the parameter change information may be set to indicate that the video content is complex. It should be understood that these are all intended to be within the scope of the present application.
Based on the above description of the code rate variation information and the parameter variation information, the step 230 includes any one of the following steps (232 to 238).
Step 232, when the code rate change information includes a code rate surge and the parameter change information indicates that the video content is simple, gradually increasing the real-time consumption code rate in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the original resolution of the current video frame is increased to obtain the target resolution.
Since human eyes are more sensitive to the picture quality change in a simple scene, the situation that the parameter change information indicates that the video content is simple and the code rate change information comprises code rate sharp increase is avoided, and the picture quality of the video is increased sharply due to overlarge resolution regulation and control force. Therefore, the embodiment of the application adopts the code rate regulation firstly aiming at the scene of simple video content and sudden increase of the code rate, and then determines whether to adopt the resolution regulation or not based on the result of the code rate regulation.
For the scene of simple video content and sudden increase of the code rate, the real-time consumption code rate is gradually increased in the target D frame interval. Alternatively, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is increased to obtain the target resolution. The mode of improving the code rate is adopted in the D frame interval, so that the picture quality of the video is improved to some extent, and smoother transition can be realized.
In this example, since the rate change information includes a sudden increase in the rate, the real-time target rate is high and the real-time consumption rate is low in the target D frame interval, and at this time, a waste of the rate may occur. To avoid the waste of the code rate, optionally, the video encoding end fills in redundant error correction packets, for example, data packets of a part of the key frames are retransmitted, so as to enhance the error correction capability of the key frames.
Step 234, in the case that the code rate variation information includes a code rate dip and the parameter variation information indicates that the video content is simple, gradually reducing the real-time consumption code rate in the target D frame interval; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the original resolution of the current video frame is reduced to obtain the target resolution.
Since human eyes are more sensitive to the picture quality change in a simple scene, the picture quality of the video is prevented from being drastically reduced due to overlarge resolution regulation and control force under the condition that the parameter change information indicates that the video content is simple and the code rate change information comprises code rate dip. Therefore, the embodiment of the application adopts the code rate regulation firstly aiming at the scene of simple video content and suddenly reduced code rate, and then determines whether to adopt the resolution regulation or not based on the result of the code rate regulation.
For the scene of simple video content and sudden drop of the code rate, the real-time consumption code rate is gradually reduced in the target D frame interval. Alternatively, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is reduced to obtain the target resolution. The mode of reducing the code rate is adopted in the D frame interval, so that the picture quality of the video is reduced to some extent, and smoother transition can be realized.
Step 236, in the case that the code rate variation information includes a code rate dip and the parameter variation information indicates that the video content is complex, gradually increasing the real-time consumption code rate within the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the original resolution of the current video frame is increased to obtain the target resolution.
Under the condition that the parameter change information indicates that the video content is complex, the actual quantization parameter after the historical video frame is encoded is larger; and the code rate change information comprises code rate sudden increase, so that the real-time target code rate is higher and the quantization parameter value is smaller. Therefore, the embodiment of the application adopts code rate regulation to improve the picture quality of the video frame by frame aiming at the scene with complex video content and suddenly increased code rate, and then determines whether resolution regulation is adopted or not based on the result of the code rate regulation.
For the scene of complex video content and sudden increase of the code rate, the real-time consumption code rate is gradually increased in the target D frame interval. Alternatively, the target D frame is the next D frame, and the value of D may be 15. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is increased to obtain the target resolution. The mode of improving the code rate is adopted in the D frame interval, so that the picture quality of the video is improved to some extent, and smoother transition can be realized.
In step 238, in the case that the code rate change information includes a code rate dip and the parameter change information indicates that the video content is complex, the original resolution of the current video frame is reduced to obtain the target resolution.
Under the condition that the parameter change information indicates that the video content is complex, the actual quantization parameter after the historical video frame is encoded is larger; and because the code rate change information comprises code rate dip, the real-time target code rate is lower and the quantization parameter value is larger. Therefore, aiming at the scene of complex video content and sudden drop of the code rate, the adaptation to the current network condition is difficult to realize through the regulation and control of the code rate.
Optionally, in the case that the code rate change information includes that the code rate does not change drastically, the code rate regulation scheme inside the encoder may be directly adopted without adopting resolution regulation, so that the video encoding end directly determines the original resolution of the current video frame as the target resolution.
In summary, according to the technical scheme provided by the embodiment of the application, the resolution regulation and control is divided into multiple types by combining the fluctuation of the network quality and the complexity of the video content. In addition, for the scene with simple video content, no matter how the network quality fluctuates, the human eyes are more sensitive to the picture quality change under the simple scene, so that the picture quality change of the video is smoothed by adopting a code rate regulation mode, and when the code rate regulation cannot adapt to the network quality, the picture quality change of the video is effectively avoided by adopting a resolution regulation mode. In addition, aiming at a scene with complex video content, when the code rate suddenly drops, the resolution is directly regulated so as to ensure the following performance of the code rate; when the code rate suddenly increases, the code rate is regulated and controlled to ensure the smoothness of the picture quality change preferentially, so that the abrupt change of the picture quality is effectively avoided. According to the embodiment of the application, through the fine resolution regulation scheme, the video playing effect or the video call effect and the like are improved, and better video playing or video call experience is brought to users.
In the resolution adjustment scheme described in the above embodiment, the quantization parameter value of the video frame previous to the current video frame is used as the measure of the complexity of the content of the current video frame, which is suitable for the situation that the difference between the previous frame and the next frame of the video is small. Based on this, it is also necessary to determine whether a scene is changed before the resolution adjustment scheme described in the above embodiment is adopted.
Based on this, in one example, the above method further comprises the following steps.
Step 250 detects a scene change of the current video content relative to the historical video content.
In the scenes such as video playing or video communication, once the scene changes, the content complexity of the video content may be greatly changed, so that a large code rate fluctuation may be caused, and the resolution regulation scheme described in the above embodiment is interfered. Therefore, the embodiment of the application can detect the scene change of the current video content relative to the historical video content first so as to avoid errors and adverse effects of the scene change on resolution regulation. The method for detecting the scene change according to the embodiment of the application is not limited, and optionally, the method for detecting the scene change comprises the following steps: key block matching algorithm, full frame matching algorithm, downsampled frame matching algorithm, etc. In general, the computational complexity of the key block matching algorithm is low. Next, a description will be given of a process of applying the key block matching algorithm to detect a scene change.
In one example, the above-described step 250 includes the following steps (252-258).
At step 252, at least one key block is determined from the current video frame.
The embodiment of the application does not limit the selected number, the selected position, the selected size and the like of the key blocks. In one example, the video encoding end determines a key blocks from the current video frame, a being a positive integer, such as 4, 5, 6, etc. In one example, at least one key block is the same size, e.g., 16 x 16 pixels each, or 8 x 8 pixels each, or 32 x 32 pixels each, or 4 x 4 pixels each, or 64 x 64 pixels each; alternatively, at least one of the key blocks may not be exactly the same size, e.g., one key block may be 16 x 16 pixels in size, another key block may be 32 x 32 pixels in size, etc. In one example, the at least one key block is uniformly distributed in the current video frame or the at least one key block is concentrated in a region in the current video frame, e.g., the at least one key frame is concentrated in a middle region of the current video frame.
Illustratively, the video encoding end determines 5 key blocks from the current video frame, the 5 key blocks are uniformly distributed in the current video frame, and the size of each of the 5 key blocks is 16×16 pixels. For example, as shown in fig. 3, 5 blocks with a size of 16×16 pixels are selected as key blocks at and around the center of the current video frame, the Guan Jiankuai of the center of the current video frame is 310, and the key blocks around the center are 320. Wherein each key block 320 is 64 pixels horizontally and vertically offset from key block 310 so that the 5 key blocks exhibit a uniform distribution in the current video frame as shown in fig. 3.
Step 254, for a first key block of the at least one key block, performing a matching block search with the target radius as a search radius at a position corresponding to the first key block in a video frame previous to the current video frame.
After the key blocks are determined, matching of the key blocks is performed. In the embodiment of the application, the key block matching is performed on the current video frame and the previous video frame of the current video frame, so that for each key block in at least one key block, the matching block searching is performed from the previous video frame of the current video frame. For a first key block of the at least one key block, a corresponding position of the first key block in a video frame preceding the current video frame needs to be determined first, and then a matching block search is performed around the corresponding position with a target radius r as a search radius. The size of the target radius is not limited in the embodiment of the present application, and alternatively, the target radius is 1 pixel, or 2 pixels, or 4 pixels, or 8 pixels.
In step 256, if the square error between the searched square blocks and the first key block is smaller than the target threshold, it is determined that the first key block has a matching block in the video frame before the current video frame.
If it can be searched that the mean square error (Mean Square Error, MSE) between a certain block and the first key block is less than the target threshold within the search range of the video frame preceding the current video frame, the block is determined to be a matching block of the first key block, i.e. a matching block of the first key block exists in the video frame preceding the current video frame. Optionally, the video coding end may use MAD (Mean Absolute Deviation, mean absolute error), SAD (Sum of Absolute Difference, sum of absolute errors) and the like to search the matching block in addition to using the mean square error as the criterion for searching the matching block. Alternatively, the sizes of the matching block and the key block need to be the same, for example, assuming that the size of the first key block is 16×16 pixels, the size of the matching block of the first key block is also 16×16 pixels.
Illustratively, the mean square error MSE between the searched square and the key block in the search range is:
wherein, N is the size of the key block, for example, N is 16; c (C) ij The pixel value corresponding to the ith row and the jth column in the key block of the current video frame; r is R ij And the pixel value corresponding to the j-th column of the ith row in the blocks searched in the search range. The smaller the MSE calculated by the above calculation formula, the higher the similarity between the searched block and the key block. In the embodiment of the application, a target threshold Th is set 0 At MSE less than Th 0 And when the searched block is regarded as a matching block of the key block.
In the embodiment of the application, under the condition that at least one key block has a matching block in the previous video frame of the current video frame, the current video frame is considered to be similar to the previous video frame, namely, the current video frame is determined to have no scene change relative to the historical video frame; and if a matching block does not exist in a video frame before the current video frame in a certain Guan Jiankuai existing in the at least one key block, determining that the current video frame has a scene change relative to the historical video frame.
Of course, in practical application, it may also be determined that the current video frame does not have a scene change relative to the historical video frame when a matching block exists in a video frame preceding the current video frame in a target number of key blocks in the at least one key block. For example, 5 key blocks are selected in the current video frame, and if 4 key blocks in the 5 key blocks have matching blocks in a video frame before the current video frame, it is determined that the current video frame has no scene change relative to the historical video frame. It should be understood that these solutions are also within the scope of the present application.
Based on the scene change detection, in one example, the step 230 is implemented as: and determining the target resolution based on the code rate change information and the parameter change information under the condition that the current video content does not change in scene relative to the historical video content. That is, when no scene change is detected, the resolution regulation scheme provided by the above embodiment is adopted, and the fluctuation of network quality and the complexity of video content are combined to determine the target resolution.
Based on the scene change detection, in another example, after the step 256, the method further includes: under the condition that the current video content changes in a scene relative to the historical video content, determining the resolution corresponding to the real-time target code rate in the target corresponding relation as target resolution; the target corresponding relation comprises at least one group of corresponding relation between target code rate and resolution. That is, when the scene is detected to change, a code table scheme is adopted to determine the target resolution, the video coding end determines the resolution corresponding to the real-time target code rate from the target corresponding relation, and the resolution is taken as the target resolution.
In summary, according to the technical scheme provided by the embodiment of the application, the scene change of the current video content relative to the historical video content is detected, and the target resolution is determined by combining the fluctuation of the network quality and the complexity of the video content under the condition that no scene change is detected, so that the influence of the code rate fluctuation generated by the scene change on the resolution regulation is effectively avoided, and the accuracy of the resolution regulation is improved.
In the following, an exemplary embodiment is described for describing a video encoding method provided by the present application.
Referring to fig. 4, a flowchart of a video encoding method according to an embodiment of the application is shown. The method can be applied to the video coding end 110 of the instant messaging scene shown in fig. 1. As shown in fig. 4, the method includes the following steps (401 to 412).
In step 401, video frame data is acquired. Under the condition of starting a video call or playing video (such as live video), the video acquisition module of the video coding end can call the camera module to acquire video frames at a target frame rate, and transmit the acquired video frame data to the audio/video processing module of the video coding end for processing.
Step 402, obtaining network quality feedback. The audio/video processing module of the video coding end can determine the real-time target code rate R by using a bandwidth estimation method and is used for indicating the current network condition. Meanwhile, this step maintains a rate buffer_r, which may be a rate buffer of a second level, for example, 5 seconds. Average value R of code rate buffer av Is a historical average code rate used to indicate historical network conditions.
Step 403, obtaining video content feedback. When the code rate is stable, the higher the texture complexity of the video frame is, the larger the actual quantization parameter value after coding is. Thus, this step uses quantization parameter values to indicate the content complexity of the video content. The video encoding end obtains a quantization parameter value of a video frame before the current video frame, namely a first quantization parameter value QP. Meanwhile, the video coding end establishes and maintains a quantization parameter value buffer_qp with a length of N, optionally, N takes a value of 10. The average value of the buffer_QP is the second quantization parameter value QP av For indicating the content complexity of the recent video content.
Step 404, it is detected whether a scene change has occurred. Once a scene change occurs, the content complexity of the video content may change significantly, possibly causing a large fluctuation in the code rate. Therefore, the scene change of the current video content relative to the historical video content is detected first, so that errors and adverse effects of the scene change on resolution regulation are avoided. The video coding end establishes a frame level buffer_pre for storing the previous video frame data of the current video frame, so that when the video coding end receives the current video frame in real time, the current video frame and the previous video frame are subjected to key block matching so as to detect whether the current video content has scene change relative to the historical video content. In the case that no scene change occurs, the following step 405 is performed; in case of a scene change, the following step 411 is performed.
Step 405, it is detected whether the network quality fluctuates. The video encoding end compares the above step 402 to obtainReal-time target code rate R and historical average code rate R of (2) av To determine if the network quality is fluctuating significantly. If the real-time target code rate R and the historical average code rate R av If the difference between the code rates is greater than the first threshold Th1, it is determined that the network quality fluctuates greatly and the code rate changes sharply, and the following step 406 is performed. If the real-time target code rate R and the historical average code rate R av If the difference between the code rates is less than or equal to the first threshold Th1, it is determined that the network quality fluctuation is small, and the original resolution of the current video frame is maintained, that is, the original resolution of the current video frame is taken as the target resolution, and the following step 412 is performed.
In step 406, the content complexity level is determined. The video encoding end compares the first quantization parameter value QP and the second quantization parameter value QP obtained in step 403 av To determine the content complexity of the video content. If the first quantization parameter value QP and the second quantization parameter value QP av The absolute value of the parameter difference value is larger than a second threshold Th2, so that the video content is indicated to be complex; if the first quantization parameter value QP and the second quantization parameter value QP av The absolute value of the parameter difference between them is smaller than the second threshold Th2, indicating that the video content is simple.
Step 407, executing the regulation scheme under the condition of simple video content and sudden increase of the code rate. At this time, the real-time consumption code rate is gradually increased in the target D frame interval. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is increased to obtain the target resolution.
Step 408, executing the regulation scheme under the condition of simple video content and sudden drop of the code rate. At this time, the real-time consumption code rate is gradually reduced in the target D frame interval. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is reduced to obtain the target resolution.
Step 409, executing a regulation scheme under the scene of complex video content and sudden increase of code rate. At this time, the real-time consumption code rate is gradually increased in the target D frame interval. If the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, the code rate regulation can adapt to the current network condition, and at the moment, the original resolution of the current video frame is determined to be the target resolution. If the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, the code rate regulation still cannot adapt to the current network condition, and the resolution regulation is continuously adopted, namely the original resolution of the current video frame is increased to obtain the target resolution.
Step 410, executing the regulation scheme under the scene of complex video content and sudden drop of the code rate. At this time, the resolution is directly regulated, that is, the original resolution of the current video frame is reduced to obtain the target resolution.
In step 411, a code table scheme is adopted to regulate and control the resolution. In the case of a scene change, the adjustment and control schemes in steps 405 to 410 are not applicable any more, and at this time, a code table scheme is adopted to select the resolution corresponding to the real-time target code rate as the target resolution. Meanwhile, in this step, the code rate buffer_r maintained in the above step 402 and the quantization parameter value buffer_qp maintained in the above step 403 are cleared.
At step 412, the current video frame is encoded with the target resolution. The video encoding terminal performs scaling processing on the current video frame based on the target resolution, and inputs the current video frame into the encoder for video encoding after the resolution of the current video frame is matched with the target resolution. Meanwhile, the video encoding end updates the frame level buffer_pre maintained in step 404.
The following are examples of the apparatus of the present application that may be used to perform the method embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the embodiments of the method of the present application.
Referring to fig. 5, a block diagram of a video encoding apparatus according to an embodiment of the application is shown. The device has the function of realizing the video coding method example, and the function can be realized by hardware or can be realized by executing corresponding software by hardware. The device can be the video coding end or can be arranged in the video coding end. The apparatus 500 may include: a code rate information acquisition module 510, a parameter information acquisition module 520, a resolution determination module 530, and a video encoding module 540.
The code rate information obtaining module 510 is configured to obtain code rate change information corresponding to a current network condition, where the code rate change information is used to indicate fluctuation of the current network condition relative to a historical network condition.
The parameter information obtaining module 520 is configured to obtain parameter variation information corresponding to a current video content, where the parameter variation information is used to indicate a content complexity of the current video content.
A resolution determining module 530, configured to determine a target resolution based on the code rate variation information and the parameter variation information.
A video encoding module 540, configured to encode the current video frame with the target resolution.
In one example, the code rate information obtaining module 510 is configured to: determining a real-time target code rate, wherein the real-time target code rate is used for indicating the current network condition; acquiring a historical average code rate, wherein the historical average code rate is used for indicating the historical network condition; and determining the code rate change information based on the real-time target code rate and the historical average code rate.
In one example, the determining the code rate variation information based on the real-time target code rate and the historical average code rate includes: determining a code rate difference between the real-time target code rate and the historical average code rate; when the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a positive number, determining that the code rate change information comprises severe change of the code rate and sudden increase of the code rate; when the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a negative number, determining that the code rate change information comprises severe change of the code rate and sudden drop of the code rate; and under the condition that the absolute value of the code rate difference value is smaller than a first threshold value, determining that the code rate change information comprises that the code rate is not changed drastically.
In one example, the parameter information obtaining module 520 is configured to: acquiring a first quantization parameter value, wherein the first quantization parameter value refers to a quantization parameter value of a video frame before the current video frame; obtaining a second quantization parameter value, wherein the second quantization parameter value refers to a mean value of quantization parameter values of at least one historical video frame; the parameter variation information is determined based on the first quantization parameter value and the second quantization parameter value.
In one example, the determining the parameter variation information based on the first quantization parameter value and the second quantization parameter value includes: determining a parameter difference between the first quantization parameter value and the second quantization parameter value; determining that the parameter variation information indicates that the video content is complex under the condition that the absolute value of the parameter difference value is larger than a second threshold value; and in the case that the absolute value of the parameter difference value is smaller than a second threshold value, determining that the parameter change information indicates that the video content is simple.
In one example, the resolution determination module 530 is configured to: under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is simple, the real-time consumption code rate is gradually increased in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution; under the condition that the code rate change information comprises code rate dip and the parameter change information indicates that the video content is simple, the real-time consumption code rate is gradually reduced in a target D frame interval; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, reducing the original resolution of the current video frame to obtain the target resolution; under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is complex, the real-time consumption code rate is gradually increased in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution; and under the condition that the code rate change information comprises code rate dip and the parameter change information indicates that the video content is complex, reducing the original resolution of the current video frame to obtain the target resolution.
In one example, the resolution determination module 530 is further configured to: and under the condition that the code rate change information comprises that the code rate is not changed drastically, determining the original resolution of the current video frame as the target resolution.
In one example, the video encoding module 540 is configured to: under the condition that the target resolution is larger than the original resolution of the current video frame, carrying out up-sampling processing on the current video frame; under the condition that the resolution of the current video frame after the up-sampling processing is matched with the target resolution, the current video frame after the up-sampling processing is encoded by adopting the target resolution; performing downsampling processing on the current video frame when the target resolution is less than the original resolution of the current video frame; and under the condition that the resolution of the current video frame after the downsampling process is matched with the target resolution, the target resolution is adopted to encode the current video frame after the downsampling process.
In one example, as shown in fig. 6, the apparatus 500 further includes: a scene change detection module 550, configured to detect a scene change of the current video content relative to the historical video content; the resolution determining module 530 is configured to determine the target resolution based on the code rate variation information and the parameter variation information when the current video content does not have a scene change relative to the historical video content.
In one example, as shown in fig. 6, the scene change detection module 550 is configured to: determining at least one key block from the current video frame; for a first key block in the at least one key block, searching for a matching block in a position corresponding to the first key block in a video frame previous to the current video frame by taking a target radius as a searching radius; if the square error between the searched square blocks and the first key block is smaller than a target threshold value, determining that a matching block exists in a video frame before the current video frame in the first key block; wherein, in the case that the at least one key block has a matching block in all preceding video frames of the current video frame, determining that the current video content has a scene change relative to the historical video content.
In one example, the resolution determination module 530 is further configured to: under the condition that the current video content changes in a scene relative to the historical video content, determining the resolution corresponding to the real-time target code rate in the target corresponding relation as the target resolution; the target corresponding relation comprises at least one group of corresponding relation between target code rate and resolution.
In summary, according to the technical scheme provided by the embodiment of the application, the resolution adopted in the current video frame coding process is determined by combining the fluctuation of the network quality and the complexity of the video content in the video coding process, so that the resolution can be regulated and controlled more accurately. Because human eyes are more sensitive to picture quality change in a simple scene, the application combines the complexity of video content when regulating and controlling the resolution, considers the sensitivity of human eyes to different content complexity of the video content, and thus, the regulation and control of the resolution is more accurate and comprehensive. In addition, the application considers the fluctuation of the network quality and the complexity of the video content simultaneously when the resolution is regulated and controlled, effectively avoids frequent switching of the resolution when the network quality fluctuation is too frequent, is beneficial to reducing the respiratory effect generated by frequent fluctuation of the network quality and improves the video watching and conversation experience of users.
It should be noted that, when the apparatus provided in the foregoing embodiment performs the functions thereof, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to perform all or part of the functions described above. In addition, the apparatus and the method embodiments provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the apparatus and the method embodiments are detailed in the method embodiments and are not repeated herein.
In an exemplary embodiment, there is also provided a computer device comprising a processor and a memory, the memory having stored therein a computer program, the computer program being loaded and executed by the processor to implement a video encoding method as described above.
In an exemplary embodiment, a computer readable storage medium is also provided, on which a computer program is stored, which computer program, when being executed by a processor, implements a video encoding method as described above.
In an exemplary embodiment, a computer program product is also provided which, when run on a computer device, causes the computer device to perform the video encoding method as described above.
The foregoing description of the exemplary embodiments of the application is not intended to limit the application to the particular embodiments disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the application.

Claims (13)

1. A method of video encoding, the method comprising:
acquiring code rate change information corresponding to a current network condition, wherein the code rate change information is used for indicating fluctuation of the current network condition relative to a historical network condition;
Acquiring parameter change information corresponding to current video content, wherein the parameter change information is used for indicating the content complexity of the current video content;
under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is simple, the real-time consumption code rate is gradually increased in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution;
under the condition that the code rate change information comprises code rate dip and the parameter change information indicates that the video content is simple, the real-time consumption code rate is gradually reduced in a target D frame interval; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, reducing the original resolution of the current video frame to obtain the target resolution;
Under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is complex, the real-time consumption code rate is gradually increased in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution;
when the code rate change information comprises code rate dip and the parameter change information indicates that the video content is complex, reducing the original resolution of the current video frame to obtain the target resolution;
and encoding the current video frame by adopting the target resolution.
2. The method of claim 1, wherein the obtaining code rate change information corresponding to the current network condition comprises:
determining a real-time target code rate, wherein the real-time target code rate is used for indicating the current network condition;
acquiring a historical average code rate, wherein the historical average code rate is used for indicating the historical network condition;
And determining the code rate change information based on the real-time target code rate and the historical average code rate.
3. The method of claim 2, wherein the determining the code rate variation information based on the real-time target code rate and the historical average code rate comprises:
determining a code rate difference between the real-time target code rate and the historical average code rate;
when the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a positive number, determining that the code rate change information comprises severe change of the code rate and sudden increase of the code rate;
when the absolute value of the code rate difference value is larger than a first threshold value and the code rate difference value is a negative number, determining that the code rate change information comprises severe change of the code rate and sudden drop of the code rate;
and under the condition that the absolute value of the code rate difference value is smaller than a first threshold value, determining that the code rate change information comprises that the code rate is not changed drastically.
4. The method according to claim 1, wherein the obtaining parameter change information corresponding to the current video content includes:
acquiring a first quantization parameter value, wherein the first quantization parameter value refers to a quantization parameter value of a video frame before the current video frame;
Obtaining a second quantization parameter value, wherein the second quantization parameter value refers to a mean value of quantization parameter values of at least one historical video frame;
the parameter variation information is determined based on the first quantization parameter value and the second quantization parameter value.
5. The method of claim 4, wherein the determining the parameter variation information based on the first quantization parameter value and the second quantization parameter value comprises:
determining a parameter difference between the first quantization parameter value and the second quantization parameter value;
determining that the parameter variation information indicates that the video content is complex under the condition that the absolute value of the parameter difference value is larger than a second threshold value;
and in the case that the absolute value of the parameter difference value is smaller than a second threshold value, determining that the parameter change information indicates that the video content is simple.
6. The method according to claim 1, wherein the method further comprises:
and under the condition that the code rate change information comprises that the code rate is not changed drastically, determining the original resolution of the current video frame as the target resolution.
7. The method of claim 1, wherein said encoding said current video frame with said target resolution comprises:
Under the condition that the target resolution is larger than the original resolution of the current video frame, carrying out up-sampling processing on the current video frame; under the condition that the resolution of the current video frame after the up-sampling processing is matched with the target resolution, the current video frame after the up-sampling processing is encoded by adopting the target resolution;
performing downsampling processing on the current video frame when the target resolution is less than the original resolution of the current video frame; and under the condition that the resolution of the current video frame after the downsampling process is matched with the target resolution, the target resolution is adopted to encode the current video frame after the downsampling process.
8. The method according to any one of claims 1 to 7, further comprising:
detecting a scene change of the current video content relative to the historical video content;
the step of determining the target resolution is performed in the event that the current video content is not subject to a scene change relative to the historical video content.
9. The method of claim 8, wherein detecting a scene change of the current video content relative to historical video content comprises:
Determining at least one key block from the current video frame;
for a first key block in the at least one key block, searching for a matching block in a position corresponding to the first key block in a video frame previous to the current video frame by taking a target radius as a searching radius;
if the square error between the searched square blocks and the first key block is smaller than a target threshold value, determining that a matching block exists in a video frame before the current video frame in the first key block;
wherein, in the case that the at least one key block has a matching block in all preceding video frames of the current video frame, it is determined that the current video content is not subject to a scene change relative to the historical video content.
10. The method of claim 8, wherein the method further comprises:
under the condition that the current video content does not change in scene relative to the historical video content, determining the resolution corresponding to the real-time target code rate in the target corresponding relation as the target resolution;
the target corresponding relation comprises at least one group of corresponding relation between target code rate and resolution.
11. A video encoding device, the device comprising:
the code rate information acquisition module is used for acquiring code rate change information corresponding to the current network condition, wherein the code rate change information is used for indicating fluctuation of the current network condition relative to the historical network condition;
the parameter information acquisition module is used for acquiring parameter change information corresponding to the current video content, wherein the parameter change information is used for indicating the content complexity of the current video content;
the resolution determining module is used for gradually increasing the real-time consumption code rate in the target D frame interval under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is simple; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution; under the condition that the code rate change information comprises code rate dip and the parameter change information indicates that the video content is simple, the real-time consumption code rate is gradually reduced in a target D frame interval; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, reducing the original resolution of the current video frame to obtain the target resolution; under the condition that the code rate change information comprises code rate sudden increase and the parameter change information indicates that the video content is complex, the real-time consumption code rate is gradually increased in the target D frame interval; if the real-time consumption code rate is larger than the real-time target code rate after exceeding the target D frame interval, determining the original resolution of the current video frame as the target resolution; if the real-time consumption code rate is smaller than the real-time target code rate after exceeding the target D frame interval, increasing the original resolution of the current video frame to obtain the target resolution; when the code rate change information comprises code rate dip and the parameter change information indicates that the video content is complex, reducing the original resolution of the current video frame to obtain the target resolution;
And the video coding module is used for coding the current video frame by adopting the target resolution.
12. A computer device comprising a processor and a memory, the memory having stored therein a computer program that is loaded and executed by the processor to implement the video encoding method of any of claims 1 to 10.
13. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the video encoding method according to any of claims 1 to 10.
CN202110779065.0A 2021-07-09 2021-07-09 Video coding method, device, equipment and storage medium Active CN113573101B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110779065.0A CN113573101B (en) 2021-07-09 2021-07-09 Video coding method, device, equipment and storage medium
PCT/CN2022/100805 WO2023279978A1 (en) 2021-07-09 2022-06-23 Video encoding method and apparatus, device, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110779065.0A CN113573101B (en) 2021-07-09 2021-07-09 Video coding method, device, equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113573101A CN113573101A (en) 2021-10-29
CN113573101B true CN113573101B (en) 2023-11-28

Family

ID=78164256

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110779065.0A Active CN113573101B (en) 2021-07-09 2021-07-09 Video coding method, device, equipment and storage medium

Country Status (2)

Country Link
CN (1) CN113573101B (en)
WO (1) WO2023279978A1 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113573101B (en) * 2021-07-09 2023-11-28 百果园技术(新加坡)有限公司 Video coding method, device, equipment and storage medium
CN115225928B (en) * 2022-05-11 2023-07-25 北京广播电视台 Multi-type audio and video mixed broadcasting system and method
CN115396729B (en) * 2022-08-26 2023-12-08 百果园技术(新加坡)有限公司 Video target frame determining method, device, equipment and storage medium
CN117014608A (en) * 2022-09-07 2023-11-07 腾讯科技（深圳）有限公司 Video stream code rate adjusting method, device, computer equipment and storage medium
CN116886880B (en) * 2023-09-08 2023-12-26 中移（杭州）信息技术有限公司 Method, device, equipment and computer program product for adjusting surveillance video
CN117440209B (en) * 2023-12-15 2024-03-01 牡丹江师范学院 Implementation method and system based on singing scene

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105635734A (en) * 2014-11-03 2016-06-01 掌赢信息科技（上海）有限公司 Self-adaptive video coding method and device based on video conversation scene
CN107659827A (en) * 2017-09-25 2018-02-02 北京小鱼易连科技有限公司 Desktop video code control system based on content analysis

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100601615B1 (en) * 1999-08-20 2006-07-14 삼성전자주식회사 Apparatus for compressing video according to network bandwidth
CN101345867B (en) * 2008-08-22 2010-06-02 四川长虹电器股份有限公司 Code rate control method based on frame complexity
CN103974060B (en) * 2013-01-31 2016-01-27 华为技术有限公司 Video quality method of adjustment and device
CN105959700B (en) * 2016-05-31 2020-04-14 腾讯科技（深圳）有限公司 Video image coding method, device, storage medium and terminal equipment
US20180063549A1 (en) * 2016-08-24 2018-03-01 Ati Technologies Ulc System and method for dynamically changing resolution based on content
WO2018216929A1 (en) * 2017-05-25 2018-11-29 Samsung Electronics Co., Ltd. Methods and systems for saving data while streaming video
CN109561310B (en) * 2017-09-26 2022-09-16 腾讯科技（深圳）有限公司 Video coding processing method, device, equipment and storage medium
CN110650370B (en) * 2019-10-18 2021-09-24 北京达佳互联信息技术有限公司 Video coding parameter determination method and device, electronic equipment and storage medium
CN113573101B (en) * 2021-07-09 2023-11-28 百果园技术(新加坡)有限公司 Video coding method, device, equipment and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105635734A (en) * 2014-11-03 2016-06-01 掌赢信息科技（上海）有限公司 Self-adaptive video coding method and device based on video conversation scene
CN107659827A (en) * 2017-09-25 2018-02-02 北京小鱼易连科技有限公司 Desktop video code control system based on content analysis

Also Published As

Publication number Publication date
WO2023279978A1 (en) 2023-01-12
CN113573101A (en) 2021-10-29

Similar Documents

Publication Publication Date Title
CN113573101B (en) Video coding method, device, equipment and storage medium
US10334289B2 (en) Efficient approach to dynamic frame size and frame rate adaptation
US8856624B1 (en) Method and apparatus for dynamically generating error correction
US11363298B2 (en) Video processing apparatus and processing method of video stream
CN110876060B (en) Code rate adjusting method and device in coding process
US20040184534A1 (en) Motion video signal encoder and encoding method
US9826260B2 (en) Video encoding device and video encoding method
JP2019501554A (en) Real-time video encoder rate control using dynamic resolution switching
CN112929747B (en) Video coding method, device and equipment based on network feedback and storage medium
US20130003571A1 (en) Quality control apparatus, moving image transmission system, quality control method, and recording medium
US20220256114A1 (en) System and method for automatically adjusting key frame quantization parameter and frame rate
US20050024486A1 (en) Video codec system with real-time complexity adaptation
CN113286146B (en) Media data processing method, device, equipment and storage medium
Menon et al. Content-adaptive variable framerate encoding scheme for green live streaming
WO2021057480A1 (en) Video coding method and video decoding method, and related apparatuses
WO2021057479A1 (en) Video coding method, video decoding method, and related devices
JP2000138930A (en) Image decoder
US20210400281A1 (en) Method for image transmitting, transmitting device and receiving device
CN116962613A (en) Data transmission method and device, computer equipment and storage medium
Nguyen et al. A QoS-adaptive framework for screen sharing over Internet
CN115378832A (en) Congestion detection method and device, streaming media transmission system, electronic equipment and medium
CN114501014A (en) Video coding parameter processing method, system, device and storage medium
WO2021248349A1 (en) Combining high-quality foreground with enhanced low-quality background
CN115442615A (en) Video coding method and device, electronic equipment and storage medium
CN112004084A (en) Code rate control optimization method and system by utilizing quantization parameter sequencing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant