CN111491201B - Method for adjusting video code stream and video frame loss processing method - Google Patents
Method for adjusting video code stream and video frame loss processing method Download PDFInfo
- Publication number
- CN111491201B CN111491201B CN202010271209.7A CN202010271209A CN111491201B CN 111491201 B CN111491201 B CN 111491201B CN 202010271209 A CN202010271209 A CN 202010271209A CN 111491201 B CN111491201 B CN 111491201B
- Authority
- CN
- China
- Prior art keywords
- code stream
- transmission rate
- video
- preset
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 230000005540 biological transmission Effects 0.000 claims abstract description 165
- 230000008859 change Effects 0.000 claims abstract description 15
- 230000000630 rising effect Effects 0.000 claims description 21
- 239000000872 buffer Substances 0.000 claims description 19
- 230000009467 reduction Effects 0.000 claims description 15
- 230000003321 amplification Effects 0.000 claims description 14
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 14
- 230000001174 ascending effect Effects 0.000 claims description 9
- 238000013016 damping Methods 0.000 claims description 5
- 230000003111 delayed effect Effects 0.000 description 7
- 230000007423 decrease Effects 0.000 description 5
- 230000015556 catabolic process Effects 0.000 description 4
- 238000006731 degradation reaction Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
- H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64746—Control signals issued by the network directed to the server or the client
- H04N21/64761—Control signals issued by the network directed to the server or the client directed to the server
- H04N21/64769—Control signals issued by the network directed to the server or the client directed to the server for rate control
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application provides a method for adjusting video code stream and a video frame loss processing method. The method for adjusting the video code stream is applied to a wireless image transmission system and comprises the following steps: acquiring a designated transmission index of a current network, wherein the designated transmission index comprises a transmission rate and identification information representing a change trend of the transmission rate; and determining whether to adjust the current code stream parameters of the video coding according to the appointed transmission index. The method ensures that the code stream parameters of video coding can adapt to the change of the transmission rate of a wireless network, and when the transmission rate of the wireless network is reduced, the data quantity of video which needs to be transmitted per second can be reduced by reducing the code stream parameters, so that the real-time performance of video data transmission is ensured.
Description
Technical Field
The present disclosure relates to the field of video transmission technologies, and in particular, to a method for adjusting a video code stream and a video frame loss processing method.
Background
Wireless image transmission systems typically include a transmitter and a receiver that are connected by a wireless network. When the system works, the shooting equipment transmits shot video data to the transmitter, the transmitter transmits the video data to the receiver through the wireless network, and the receiver outputs the video data to the display equipment, so that the remote wireless transmission of the video data is realized.
However, when the environment of the wireless network is in dynamic change, delay occurs in the transmission of video data when the environment of the wireless network is poor, and the video playing cannot be kept in real time.
Disclosure of Invention
In order to overcome the problems in the related art, the application provides a method for adjusting a video code stream and a video frame loss processing method.
According to a first aspect of embodiments of the present application, there is provided a method for adjusting a video bitstream, applied to a wireless image transmission system, including:
acquiring a designated transmission index of a current network, wherein the designated transmission index comprises a transmission rate and identification information representing a change trend of the transmission rate;
and determining whether to adjust the current code stream parameters of the video coding according to the appointed transmission index.
As an alternative embodiment, the method further comprises:
if the identification information represents that the transmission rate is in an ascending trend and the current transmission rate is smaller than a first preset value, the current code stream parameters are maintained.
As an alternative embodiment, the method further comprises:
if the identification information represents that the transmission rate is in an ascending trend and the current transmission rate is larger than the first preset value, increasing the current code stream parameter according to preset amplification.
As an optional implementation manner, the increasing the current code stream parameter according to the preset increase includes:
if the rising amplitude of the transmission rate is larger than the rising amplitude threshold value, increasing the current code stream parameters according to the first preset rising amplitude; if the rising amplitude of the transmission rate is smaller than or equal to the rising amplitude threshold, increasing the current code stream parameter according to a second preset rising amplitude; wherein the first preset amplification is greater than the second preset amplification.
As an alternative embodiment, the method further comprises:
if the current code stream parameter is determined to be increased, determining that the increased code stream parameter does not exceed the preset upper limit value of the code stream parameter;
if the identification information represents that the transmission rate is in a descending trend, and the current transmission rate is larger than a second preset value, keeping the current code stream parameters; the second preset value is a transmission rate of the code stream corresponding to the upper limit value of the code stream parameter.
As an alternative embodiment, the method further comprises:
if the identification information represents that the transmission rate is in a descending trend and the current transmission rate is smaller than the second preset value, reducing the current code stream parameters according to preset reduction.
As an optional implementation manner, the reducing the current code stream parameter according to the preset reduction includes:
if the descending amplitude of the transmission rate is larger than the descending amplitude threshold, reducing the current code stream parameters according to the first preset reduction; if the descending amplitude of the transmission rate is smaller than or equal to the descending amplitude threshold, reducing the current code stream parameter according to a second preset descending amplitude; wherein the first preset damping is greater than the second preset damping.
According to a second aspect of embodiments of the present application, there is provided a video frame loss processing method, applied to a wireless image transmission system, including:
when receiving a video frame sent by a transmitter, determining the current total delay frame number; if the current total delay frame number is determined to be larger than the preset delay upper limit, determining to perform frame loss processing on the video data sent by the transmitter; the code stream parameters corresponding to the video data sent by the transmitter are determined according to the appointed transmission index of the current network, and the appointed transmission index comprises a transmission rate and identification information representing the change trend of the transmission rate.
As an optional implementation manner, the determining to perform frame loss processing on the video data sent by the transmitter includes:
determining the frame loss number; the frame loss number is the difference value between the total delay frame number and the delay upper limit;
and discarding one or more received video frames with the last time sequence in the group of pictures GOP according to the determined frame loss number.
As an optional implementation manner, the determining the current total delay frame number includes:
determining a first delay frame number according to the time interval of receiving video frames for two times;
determining a second delay frame number according to the current frame buffer number;
determining a total delay frame number; the total delay frame number is a sum of the first delay frame number and the second delay frame number.
The technical scheme provided by the embodiment of the application can comprise the following beneficial effects:
in the embodiment of the application, whether to adjust the current code stream parameters of the video coding is determined according to the acquired designated transmission index of the current network, so that the code stream parameters of the video coding can adapt to the change of the transmission rate of the wireless network. Therefore, when the wireless network signal is poor and the transmission rate is reduced, the data volume required to be transmitted by the video per second can be reduced by reducing the code stream parameter, so that the real-time performance of video data transmission is ensured.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic view of an application scenario of a wireless image transmission system according to an exemplary embodiment of the present application.
Fig. 2 is a flowchart illustrating a method of adjusting a video bitstream according to an exemplary embodiment of the present application.
Fig. 3 is a flowchart of a video frame loss processing method according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.
Next, embodiments of the present application will be described in detail.
Referring to fig. 1, fig. 1 is a schematic view of an application scenario of a wireless image transmission system according to an exemplary embodiment of the present application.
A wireless image transmission system generally includes a transmitter 1 and a receiver 2, and a connection is established between the transmitter 1 and the receiver 2 through a wireless network. Wherein the transmitter 1 may be connected to a photographing device and the receiver 2 may be connected to a display device.
When the system works, shooting equipment transmits shot video data to the transmitter 1, the transmitter 1 transmits the video data to the receiver 2 through a wireless network, and the receiver 2 outputs the video data to display equipment, so that remote wireless transmission of the video data is realized.
But the environment of the wireless network is dynamically changing, so the wireless network signal is not necessarily stable. When the wireless network signal is poor, the transmission rate of the wireless network is also reduced, and if the transmission rate at the moment cannot meet the code stream parameters of the current video coding, the video is played in a blocking way, and the video picture is not real-time any more.
In order to solve the above-mentioned problems, the present application provides a method for adjusting a video code stream, which is applied to a wireless image transmission system and can be executed by a transmitter in the wireless transmission system. Turning to fig. 2, fig. 2 is a flowchart illustrating a method for adjusting a video bitstream according to an exemplary embodiment of the present application. The method comprises the following steps:
step 201, acquiring a designated transmission index of a current network.
Wherein the designated transmission index includes a transmission rate and identification information characterizing a trend of change in the transmission rate.
Step 202, determining whether to adjust current code stream parameters of video coding according to the specified transmission index.
Since the environment of the wireless network may change dynamically, the transmission rate of the wireless network is also in the process of changing dynamically. In order to determine the trend of the transmission rate, the real-time transmission rate of the wireless network may be obtained according to the set frequency. In one example, after determining the transmission rate at the current time, the transmission rate at the current time may be compared with the transmission rate determined at the previous time, so as to determine whether the transmission rate is in an upward trend or a downward trend.
The trend of the transmission rate can be characterized by the identification information. For example, in one implementation, it may be set that the identification information equal to 1 indicates that the transmission rate is in an upward trend, and that the identification information equal to 0 indicates that the transmission rate is in a downward trend. Of course, the foregoing is merely an example, and the skilled person may individually select other identification information, which is not limited in this application.
By analyzing the content of the identification information, the trend of the change in the transmission rate can be determined. In one case, if the transmission rate tends to increase, the stream parameters of the video coding may be selected to increase. However, if the code stream parameter is adjusted as soon as the transmission rate increases, the frequency of the change of the code stream parameter may be too frequent. Because the code stream parameters and the image quality of the video have a direct association relationship, the code stream parameters are frequently changed, and the image quality of the video is also frequently changed, so that the image quality of the video is unstable for users, and the appearance of the users is greatly reduced.
Therefore, to solve the above problem, a threshold value of the transmission rate, which may be referred to as a first preset value, may be set for the case where the transmission rate rises. If the transmission rate is in an ascending trend, but the current transmission rate is still smaller than the first preset value, the current transmission rate is still lower, and even if the code stream parameter is increased, the video image quality is not obviously improved, so that the stability of the video image quality is ensured, the current code stream parameter can be kept unchanged, and adjustment is not performed.
Correspondingly, if the transmission rate is in an ascending trend and the current transmission rate is greater than the first preset value, the code stream parameter is increased, the video image quality can be obviously improved, and therefore the current code stream parameter can be selectively increased.
However, it is obvious that the code stream parameters cannot be increased without any limit when the transmission rate increases. The upper limit values of the code stream parameters are different corresponding to different transmission rates. Therefore, an upper limit value of the code stream parameter can be set, and when the current code stream parameter is increased, the code stream parameter is ensured not to exceed the upper limit value of the code stream parameter.
In addition, from another perspective, when the image quality of the video reaches a certain definition, the requirements of most users can be met, and further increasing the code stream parameters to improve the image quality has not great significance. Therefore, further, when the upper limit value of the code stream parameter is set, the code stream parameter can be relatively conservative, so that the code stream parameter is kept at a certain distance from the code stream which can be supported by the highest transmission rate of the wireless network. For example, the highest transmission rate of the wireless network may support a 18M code stream parameter, and the upper limit value of the code stream parameter may be set to 12M, because the video image quality corresponding to the 12M code stream parameter is sufficiently clear.
By the arrangement, the number of times of increasing the code stream parameter when the transmission rate rises can be limited, the condition that the code stream parameter needs to be reduced is reduced, and the effects of reducing the overall adjustment number of times of the code stream parameter and stabilizing the video image quality can be achieved. Specifically, if the upper limit value of the code stream parameter is different from the code stream supportable by the highest transmission rate by a certain value, the transmission rate is reduced, and the code stream parameter does not need to be reduced.
For example, continuing with the example above, the highest transmission rate of the wireless network may support a 18M code stream parameter, the upper limit value of which is set to 12M. Then, when the transmission rate increases to a level that can support a code stream greater than 12M, the code stream parameter will not be increased any more, and the code stream parameter will be maintained at a value of 12M or close to 12M. When the transmission rate is reduced, for example, from 18M to 15M, or from 16M to 14M, the code stream parameter may not be reduced as long as the reduced transmission rate is still greater than the second preset value. The second preset value may support the transmission rate of the code stream corresponding to the upper limit value of the code stream parameter, and in the above example, may support the transmission rate of the 12M code stream.
Further, it is considered that when the code stream parameter of video coding is increased or decreased, if the amplitude of one adjustment is too large, the code stream parameter is changed too much, the image quality of the video will suddenly become blurred or clear, and the stability of the video image quality is poor.
Therefore, for increasing the code stream parameter, one increase may be set in advance. When the identification information indicates that the transmission rate is in an ascending trend and the current transmission rate is greater than a first preset value, the current code stream parameter can be increased according to the preset amplification. For example, in one specific example, the code stream parameter may be increased with an increase of 2M per second, and in another specific example, the code stream parameter may be increased with an increase of 0.5M per second.
For reducing the code stream parameters, a damping can also be preset. When the identification information represents that the transmission rate is in a descending trend and the current transmission rate is smaller than a second preset value, the current code stream parameters can be reduced according to preset reduction. For example, in one specific example, the code stream parameter may be reduced at a reduction of 2M per second, and in another specific example, the code stream parameter may be reduced at a reduction of 0.5M per second.
Since the transmission rate of the wireless network may vary greatly or not so, if the adjustment amplitude of the code stream parameter is fixed, some problems may occur. For example, if the current code stream parameter is 2M and the preset increase is 0.5M per second, if the transmission rate is suddenly and greatly increased to a level that can support 12M code stream at the next moment, at this time, if the code stream parameter is increased by 0.5M per second according to the preset increase, a long period of time is required to increase the code stream parameter to 12M, so that the transmission rate of the current network cannot be maximized at the fastest speed.
Another possibility is that if the preset reduction is 0.5M per second, when the transmission rate suddenly drops significantly, for example, the transmission rate drops from a 12M capable stream to a 2M capable stream, the video will be jammed for a considerable period of time if the stream parameters are reduced by a reduction of 0.5M per second.
If the preset increase or decrease is too large, the adjustment will also be problematic when the transmission rate changes little by little. For example, if the preset reduction is 2M per second, when the transmission rate is reduced from a 6M supportable code stream to a 5.5M supportable code stream, if the current code stream parameter is 6M, the code stream parameter needs to be reduced, and after the reduction by 2M per second, the code stream parameter is 4M, so that the transmission rate cannot be utilized maximally. If the current code stream parameter is 6M, the preset amplification is 2M per second, and when the transmission rate is increased from the code stream capable of supporting 6M to the code stream capable of supporting 7.5M, the code stream parameter cannot be increased by adopting the amplification of 2M per second, and similarly, the transmission rate cannot be utilized maximally.
For this reason, the present application provides an embodiment in which a plurality of different increases can be provided for the increase in the transmission rate to cope with the different increases in the transmission rate. In an alternative embodiment, a rise amplitude threshold may be preset. When the transmission rate rises, the current transmission rate and the transmission rate determined at the last time or the last time are used for making a difference to calculate the rising amplitude of the transmission rate. If the calculated rise amplitude is greater than the rise amplitude threshold, a larger increase may be selected to increase the current code stream parameter, which may be referred to as a first predetermined increase, such as 2M per second.
If the calculated rise amplitude is less than or equal to the rise amplitude threshold, a smaller rise may be selected to increase the current code stream parameter, which may be referred to as a second predetermined rise, for example, by 0.5M per second.
Of course, to more finely cope with different changes of the transmission rate, two different rising amplitude thresholds may be further set, and three different preset increases may be set, which is not specifically limited in this application.
Accordingly, for the decrease of the transmission rate, a plurality of different reductions may be set to cope with the different decrease magnitudes of the transmission rate. In an alternative embodiment, a drop amplitude threshold may be preset. When the transmission rate decreases, the difference between the transmission rate determined at the previous or previous time and the current transmission rate (or the absolute value of the current transmission rate subtracted from the previous transmission rate) may be used to calculate the decrease amplitude of the transmission rate. If the calculated drop amplitude is greater than the drop amplitude threshold, a larger step-down amplitude may be selected to reduce the current code stream parameter, which may be referred to as a first predetermined step-down amplitude, such as a step-down of 2M per second.
If the calculated degradation is less than or equal to the degradation threshold, a smaller degradation may be selected to reduce the current code stream parameter, which may be referred to as a second predetermined degradation, such as a reduction of 0.5M per second.
It will be appreciated that for ease of characterizing the transmission rate of the wireless network, in an alternative embodiment, a modulation and coding scheme MCS may also be employed. Different MCS levels correspond to different transmission rates, for example, the MCS levels may include MCS0 to MCS15, and when the determined MCS level is higher, the transmission rate of the current network is also higher; when the MCS level changes, the transmission rate of the current network also changes. Whether the transmission rate is rising or falling can be determined by a change in MCS level.
The foregoing is a detailed description of a method for adjusting a video bitstream provided herein. According to the method, whether the current code stream parameters of the video coding are adjusted can be determined according to the acquired appointed transmission index of the current network, so that the code stream parameters of the video coding can adapt to the change of the transmission rate of the wireless network. Therefore, when the wireless network signal is poor and the transmission rate is reduced, the data volume required to be transmitted by the video per second can be reduced by reducing the code stream parameter, so that the real-time performance of video data transmission is ensured. When the transmission rate of the wireless network is increased, the code stream parameter can be correspondingly increased, so that the bandwidth is utilized to the maximum extent, and a clearer video image quality is provided for a user.
Further, during transmission of video data, video frames are transmitted by the transmitter to the receiver over the wireless network. But the wireless network is not necessarily stable, and when the wireless network signal is degraded, transmission of video frames will be blocked. For a receiver, it will not be able to receive a video frame at some time when it should be, resulting in the playback of the video frame not being in real time.
To facilitate understanding of the above problems, a specific example is given below.
For example, in the normal case of a wireless network, a receiver receives a video frame at regular intervals. Each point in time at which a video frame should be received may be referred to as a time instant, at a first time instant, the receiver may receive a first frame that will be played during the play period of the next frame; at a second time, the receiver may receive a second frame that will be played during the play period of the next frame.
However, if the wireless network signal is degraded at the third time, the transmission of the video frame is blocked, the receiver will not receive the third frame at the third time, and will only continue to play the previously received second frame during the play period of the next frame. If such a situation continues for several moments, the playing of the video frames will have a non-negligible delay compared to the pictures of the real scene, and no real-time video images can be presented to the user.
Therefore, the present application solves the above-mentioned problems, and further improvements can be made on the basis of the above-mentioned method for adjusting video code stream, so as to better provide real-time video images for users.
Specifically, referring to fig. 3, fig. 3 is a flowchart of a video frame loss processing method according to an exemplary embodiment of the present application. The method is equally applicable to wireless image transmission systems, where the actions involved may be performed by a receiver. The method comprises the following steps:
step 301, when receiving a video frame sent by a transmitter, determining a current total delay frame number.
Step 302, determining whether the determined current total delay frame number is greater than a preset delay upper limit, if yes, executing step 303, and if no, ending the flow.
Step 303, determining to perform frame loss processing on the video data sent by the transmitter.
It should be noted that, the code stream parameters corresponding to the video data transmitted by the transmitter may be determined according to the specified transmission index of the current network. Wherein the designated transmission index includes a transmission rate and identification information characterizing a trend of change in the transmission rate. The specific implementation of this portion of content may refer to the foregoing detailed description of a method for adjusting a video bitstream, which is not described herein.
According to the video frame loss processing method, under the condition that the playing of the video frames is delayed, certain video frames in video data are discarded, so that the playing of the video frames can be restored to be real-time.
The frame loss processing is to restore the playing of the video frame to be real-time, and whether the playing of the video frame is real-time or not needs to be defined in real time. Because of the limited number of human visual frames, when the video frames are played with only a delay of a few frames, it is likely that the picture delay will not be perceived by the user, i.e. the video picture will be considered to be still real-time for the user.
Therefore, a delay upper limit may be preset, and when the determined current total delay frame number is greater than the delay upper limit, it is determined that frame loss processing needs to be performed on the video data sent by the transmitter.
There are a number of possible ways to determine the total delay frame number. In an alternative embodiment, after determining the next video frame to be played, the determined next video frame to be played and the next video frame to be played may be compared in time sequence, thereby determining the total delay frame number. The next video frame that should be played, i.e., the next video frame that should be played when the network is not delayed. In order to determine the timing of the video frame, the display time stamp PTS of the video frame may be used when performing timing comparison of the video frame. Of course, a GOP number may be given to a video frame in one group of pictures GOP (Group of pictures), and the GOP number of the video frame corresponds to the timing thereof. In order to obtain the PTS or GOP sequence number conveniently, when the transmitter encodes video data, the corresponding PTS or GOP sequence number may be written in a specific position such as the head of a video frame, so that when the receiver receives the video frame, the receiver may directly parse the data in the specific position to obtain the PTS or GOP sequence number of the video frame.
In another optional implementation manner, when determining the current total delay frame number, the first delay frame number may be determined according to a time interval between two times of receiving video frames before and after; determining a second delay frame number according to the current frame buffer number; and calculating the sum of the first delay frame number and the second delay frame number to obtain the current total delay frame number.
For ease of understanding, a specific example is set forth below. Referring to table 1, a total of 10 time instants are presented. At the 1 st moment, the receiver receives the 1 st frame, and at the moment, the video frame is not stored in the frame buffer, and the next playing of the 1 st frame is determined. At the 2 nd moment, the receiver receives the 2 nd frame, and at the moment, the 1 st frame received at the previous moment is stored in the frame buffer memory, and the next playing of the 2 nd frame is determined. At time 3, the wireless network is blocked, the receiver cannot successfully receive the 3 rd frame, and the 2 nd frame received at the previous time is stored in the frame buffer, so that only the 2 nd frame which is played next can be determined. At time 4, the wireless network is not yet restored, and the situation is the same as at time 3.
At time 5, the wireless network returns to normal, the receiver receives frames 3, 4 and 5 once, and at the moment, the frame buffer still stores the frame 2, and since the frame 3 has been received, the next frame 3 can be determined to be played. At time 6, the receiver receives the 6 th frame, and at this time, the 3 rd, 4 th and 5 th frames are stored in the frame buffer, and it is determined that the 4 th frame is played next. At the 7 th moment, the wireless network is blocked again, the receiver cannot successfully receive the 7 th frame, and at the moment, the 4 th, 5 th and 6 th frames are stored in the frame buffer, so that the next video frame on the time sequence, namely the 5 th frame, can be continuously played, and therefore the next frame is determined to be played. At the 8 th moment, the wireless network is not recovered yet, the video frame is not received, the 5 th and 6 th frames are stored in the frame buffer, and the next playing of the 6 th frame is determined.
At the 9 th moment, the wireless network is recovered to be normal, the receiver receives the 7 th, 8 th and 9 th frames at one time, at this moment, the 6 th frame is stored in the frame buffer, and the 7 th frame is determined to be played next.
TABLE 1
When determining the total delay number, for each time when a video frame is received, the first delay frame number may be determined according to the time interval between two times of receiving the video frame. For example, when the wireless network signal is good, the time interval between two times of receiving video frames is 10ms, but at the 5 th time, the time interval from the 2 nd time to the 5 th time is 30ms, the first delay frame number is determined to be 2.
As for the number of frame buffers, in the case where the wireless network is normal, the number of video frames stored in the frame buffer should be 1 except for time 1. If the number of frame buffers is greater than 1, it indicates that a network congestion occurred at some point before. And the second delay frame number can be calculated by subtracting 1 from the frame buffer number in specific calculation. At time 5, the number of frame buffers is 1, so the second delay frame number is 0. The total delay frame number is the sum of the first delay frame number and the second delay frame number, and therefore, at time 5, the total delay frame number is 2.
As another example, at time 6, in the same manner, the first delay frame number is 0, the second delay frame number is 2, and the total delay frame number is 2.
After the total delay frame number is greater than the delay upper limit and the frame loss processing is determined to be performed on the video data sent by the transmitter, the frame loss number needs to be determined first. The number of frames lost may be determined in various ways, and in one implementation, the number of frames lost may be calculated according to a total delay frame number and a delay upper limit, and specifically, the number of frames lost may be a difference between the total delay frame number and the delay upper limit. Thus, after a frame loss, the video is actually delayed, but the number of frames delayed is equal to the upper delay limit, and still considered to be real-time. In another implementation, when it is determined that the frame loss processing is required, the total delay frame number may be directly taken as the frame loss number. Thus, after a frame loss, the video will become a 0 delay.
And after the frame loss number is determined, it is necessary to determine which frames are specifically lost. Because the encoding processing modes of different video frames are different, for an I frame, an intra-frame encoding mode is adopted, so that a complete image can be independently decoded, and for a P frame or a B frame, an inter-frame encoding mode is adopted, and other frames are needed to be relied on to decode the image. Thus in selecting frames to discard, a preferred embodiment is to select the last video frame or frames in the GOP to discard.
Note that one group GOP (Group of pictures), i.e., the distance between two I frames, forms one group of pictures between two I frames as one GOP. Since video frames other than I frames generally follow forward temporal prediction, i.e., the decoded playback of the video frame depends on the previous video frame, the playback of video frames for non-I frames generally can only be performed in time sequence, and no skip can occur. For example, in the foregoing example, although the 3 rd, 4 th and 5 th frames are received once at the 5 th time, the 5 th frame which is the most real time cannot be directly played, because the playing of the 5 th frame depends on the decoding playing of the 4 th frame, and the playing of the 4 th frame depends on the decoding playing of the 3 rd frame, so that it can only play the next video frame in time sequence, namely, the 3 rd frame.
Therefore, when selecting a video frame that can be discarded, to ensure the stability of the video picture, a video frame that is later in time in the GOP should be selected for discarding. If the number of frames dropped in one GOP is still less than the determined number of dropped frames, the dropping can continue in the next GOP.
An example is provided below to aid understanding.
See table 2. For example, in the transmission of the video frame, from the time 2, the wireless network is blocked, and the wireless network is not recovered until the time 7, and then at the time 7, the receiver will receive the 2 nd, 3 rd, 4 th, 5 th, 6 th and 7 th frames at one time, and the frame buffer stores the 1 st frame, and according to the above determination method, the total delay frame number can be determined to be 5. If the delay upper limit is set to 1, the number of lost frames can be determined to be 4.
TABLE 2
It is assumed that ten video frames are contained in one GOP, i.e., 1 st to 10 th frames are one GOP. If it is considered that a maximum of four video frames can be discarded in one GOP, i.e., frames 7, 8, 9, 10 in one GOP are considered to be discardable. Thus, in the above example, the 7 th frame received at the 7 th time may be directly discarded, and the other 2 nd, 3 rd, 4 th, 5 th, and 6 th frames may be stored in the frame buffer. At the 8 th moment, if the wireless network keeps normal, the 8 th frame which is successfully received can also be directly discarded; correspondingly, the 9 th frame and the 10 th frame received at the 9 th and the 10 th times can be directly discarded. As shown in table 3 below.
It can be seen that, at the 12 th time of the second GOP, the 7 th, 8 th, 9 th and 10 th frames of the first GOP are skipped and the 1 st frame of the next GOP is directly used as the next video frame to be played, so that the number of delayed frames can be reduced, the number of delayed frames of the video is equal to the preset upper delay limit, and the video playing is restored to real time.
TABLE 3 Table 3
If it is considered that at most two video frames can be discarded in one GOP, i.e., the 9 th frame and the 10 th frame in one GOP are considered to be discarded. Then, in the above example, all video frames received at time 7 may be stored in the frame buffer, and the first frame may not be lost until the 9 th frame is received at time 9, and the second frame may be lost when the 10 th frame is received at time 10. It is not difficult to find that only two frames are discarded in the current GOP, and two more frames are required to be discarded according to the determined number of frames to be discarded. Two frames that need to be discarded again can be discarded at the next GOP, specifically, when the 9 th frame and the 10 th frame of the next GOP are received, thereby completing the required frame loss.
The number of frames that can be discarded at most in one GOP can be limited by presetting a value. For example, a preset limit may be set, corresponding to the above example, where the preset limit may be set equal to 2, and if the determined frame loss number is greater than the preset limit (corresponding to the above example, i.e. 4>2), a part of the frame loss operation (i.e. 2 frame loss tasks exceeding) is allocated to be performed in the next GOP.
In the video frame loss processing method, when video playing is delayed, frame loss processing is carried out on specific video frames in video data, so that the discarded video frames can be skipped to be not played on the basis of affecting the stability of video pictures as little as possible, the video playing delay is reduced, and the video playing is recovered in real time.
The foregoing describes specific embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
The foregoing description of the preferred embodiments of the present invention is not intended to limit the invention to the precise form disclosed, and any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A method for adjusting video code streams, applied to a wireless image transmission system, comprising:
acquiring a designated transmission index of a current network, wherein the designated transmission index comprises a transmission rate and identification information representing a change trend of the transmission rate;
determining whether to adjust current code stream parameters of video coding according to the appointed transmission index, wherein if the identification information represents that the transmission rate is in an ascending trend and the current transmission rate is smaller than a first preset value, the current code stream parameters are maintained; if the identification information represents that the transmission rate is in an ascending trend and the current transmission rate is greater than the first preset value, increasing the current code stream parameter according to preset amplification;
the step of increasing the current code stream parameters according to the preset amplification comprises the following steps:
if the rising amplitude of the transmission rate is larger than the rising amplitude threshold value, increasing the current code stream parameters according to the first preset rising amplitude; if the rising amplitude of the transmission rate is smaller than or equal to the rising amplitude threshold, increasing the current code stream parameter according to a second preset rising amplitude; wherein the first preset amplification is greater than the second preset amplification.
2. The method of adjusting a video bitstream according to claim 1, further comprising:
if the current code stream parameter is determined to be increased, determining that the increased code stream parameter does not exceed the preset upper limit value of the code stream parameter;
if the identification information represents that the transmission rate is in a descending trend, and the current transmission rate is larger than a second preset value, keeping the current code stream parameters; the second preset value is a transmission rate of the code stream corresponding to the upper limit value of the code stream parameter.
3. The method of adjusting a video bitstream according to claim 2, further comprising:
if the identification information represents that the transmission rate is in a descending trend and the current transmission rate is smaller than the second preset value, reducing the current code stream parameters according to preset reduction.
4. The method for adjusting a video bitstream according to claim 3, wherein said reducing current bitstream parameters by a preset reduction comprises:
if the descending amplitude of the transmission rate is larger than the descending amplitude threshold, reducing the current code stream parameters according to the first preset reduction; if the descending amplitude of the transmission rate is smaller than or equal to the descending amplitude threshold, reducing the current code stream parameter according to a second preset descending amplitude; wherein the first preset damping is greater than the second preset damping.
5. The video frame loss processing method is characterized by being applied to a wireless image transmission system and comprising the following steps of:
when receiving a video frame sent by a transmitter, determining the current total delay frame number; if the current total delay frame number is determined to be larger than the preset delay upper limit, determining to perform frame loss processing on the video data sent by the transmitter; the method comprises the steps that a code stream parameter corresponding to video data sent by a transmitter is determined according to a specified transmission index of a current network, the specified transmission index comprises a transmission rate and identification information representing a change trend of the transmission rate, and if the identification information represents that the transmission rate is an ascending trend, the current transmission rate is smaller than a first preset value, and the current code stream parameter is kept; if the identification information represents that the transmission rate is in an ascending trend and the current transmission rate is greater than the first preset value, increasing the current code stream parameter according to preset amplification; the step of increasing the current code stream parameters according to the preset amplification comprises the following steps:
if the rising amplitude of the transmission rate is larger than the rising amplitude threshold value, increasing the current code stream parameters according to the first preset rising amplitude; if the rising amplitude of the transmission rate is smaller than or equal to the rising amplitude threshold, increasing the current code stream parameter according to a second preset rising amplitude; wherein the first preset amplification is greater than the second preset amplification.
6. The method for processing video frame loss according to claim 5, wherein said determining to perform frame loss processing on video data transmitted by the transmitter comprises:
determining the frame loss number; the frame loss number is the difference value between the total delay frame number and the delay upper limit;
and discarding one or more received video frames with the last time sequence in the group of pictures GOP according to the determined frame loss number.
7. The method for processing video frame loss according to claim 5, wherein said determining a current total delay frame number comprises:
determining a first delay frame number according to the time interval of receiving video frames for two times;
determining a second delay frame number according to the current frame buffer number;
determining a total delay frame number; the total delay frame number is a sum of the first delay frame number and the second delay frame number.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010271209.7A CN111491201B (en) | 2020-04-08 | 2020-04-08 | Method for adjusting video code stream and video frame loss processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010271209.7A CN111491201B (en) | 2020-04-08 | 2020-04-08 | Method for adjusting video code stream and video frame loss processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111491201A CN111491201A (en) | 2020-08-04 |
| CN111491201B true CN111491201B (en) | 2023-04-25 |
Family
ID=71798394
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010271209.7A Active CN111491201B (en) | 2020-04-08 | 2020-04-08 | Method for adjusting video code stream and video frame loss processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111491201B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112399268A (en) * | 2020-11-12 | 2021-02-23 | 唐桥科技(杭州)有限公司 | Real-time streaming media transmission method and device and electronic equipment |
| CN113315773B (en) * | 2021-05-31 | 2022-08-12 | 浙江大华技术股份有限公司 | Code rate adjusting method and device, electronic equipment and storage medium |
| CN114125453A (en) * | 2021-11-23 | 2022-03-01 | 江苏云从曦和人工智能有限公司 | Video transmission method, system, device and medium |
| CN114422866B (en) * | 2022-01-17 | 2023-07-25 | 深圳Tcl新技术有限公司 | Video processing method and device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102036024A (en) * | 2009-10-07 | 2011-04-27 | 索尼公司 | Transmission apparatus and transmission method |
| CN109004972A (en) * | 2018-07-13 | 2018-12-14 | 深圳市道通智能航空技术有限公司 | Data transmission method, device, system and the surface map transmission module of UAV system |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8281351B2 (en) * | 2005-04-29 | 2012-10-02 | Alcatel Lucent | System, method, and computer readable medium rapid channel change |
| JP2007180600A (en) * | 2005-12-26 | 2007-07-12 | Toshiba Corp | Electronic equipment and method of controlling data transfer rate of electronic equipment |
| CN101159862B (en) * | 2007-11-29 | 2010-06-09 | 北京中星微电子有限公司 | Frame rate control method and device |
| CN101360058A (en) * | 2008-09-08 | 2009-02-04 | 华为技术有限公司 | Method and apparatus for cache overflow control |
| CN102325274B (en) * | 2011-10-13 | 2013-08-21 | 浙江万里学院 | Network bandwidth-adaptive video stream transmission control method |
| CN103873382A (en) * | 2012-12-17 | 2014-06-18 | 马维尔国际有限公司 | Data frame buffer method and equipment |
| CN103686449B (en) * | 2013-12-31 | 2018-01-30 | 快创科技（大连）有限公司 | It is a kind of to lift video fluency and the caching method of image quality |
| CN104702968B (en) * | 2015-02-17 | 2019-06-11 | 华为技术有限公司 | A kind of video frame frame losing method and video transmission device |
| CN105611309B (en) * | 2015-12-22 | 2019-03-05 | 北京奇虎科技有限公司 | A kind of video transmission method and device |
| CN106303559A (en) * | 2016-08-18 | 2017-01-04 | 北京奇虎科技有限公司 | A kind of method controlling live video stream and direct broadcast server |
| CN106954101B (en) * | 2017-04-25 | 2020-04-28 | 华南理工大学 | Frame loss control method for low-delay real-time video streaming media wireless transmission |
| CN107276910A (en) * | 2017-06-07 | 2017-10-20 | 上海迪爱斯通信设备有限公司 | The real-time adjusting apparatus of video code rate and system, video server |
| CN107438031A (en) * | 2017-08-07 | 2017-12-05 | 成都三零凯天通信实业有限公司 | The audio/video flow transfer control method and system of multichannel network bandwidth adaptive |
| CN110933380A (en) * | 2019-12-17 | 2020-03-27 | 深圳市道通智能航空技术有限公司 | Image transmission control method and system and unmanned aerial vehicle |
-
2020
- 2020-04-08 CN CN202010271209.7A patent/CN111491201B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102036024A (en) * | 2009-10-07 | 2011-04-27 | 索尼公司 | Transmission apparatus and transmission method |
| CN109004972A (en) * | 2018-07-13 | 2018-12-14 | 深圳市道通智能航空技术有限公司 | Data transmission method, device, system and the surface map transmission module of UAV system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111491201A (en) | 2020-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111491201B (en) | Method for adjusting video code stream and video frame loss processing method | |
| US8351513B2 (en) | Intelligent video signal encoding utilizing regions of interest information | |
| Lee et al. | Scalable rate control for MPEG-4 video | |
| CN100401782C (en) | Method and apparatus for controlling rate of video sequence, video encoding device | |
| EP0771120B1 (en) | Video encoding apparatus | |
| US8477844B2 (en) | Method and apparatus for transmitting video | |
| US6707852B1 (en) | Digital video signal encoder and encoding method | |
| US6873654B1 (en) | Method and system for predictive control for live streaming video/audio media | |
| JP5351040B2 (en) | Improved video rate control for video coding standards | |
| US20100309987A1 (en) | Image acquisition and encoding system | |
| US20130304934A1 (en) | Methods and systems for controlling quality of a media session | |
| CN110769296B (en) | Video code rate self-adaptive adjusting mode based on local cache during transmission | |
| KR20070111550A (en) | Quasi-constant-quality rate control with look-ahead | |
| WO2007118160A1 (en) | Temporal quality metric for video coding | |
| CN111901635A (en) | Video processing method, device, storage medium and equipment | |
| JP2005322995A (en) | Buffer control method in real-time video image transfer, transmitting terminal, receiving terminal, video image distributing system and program | |
| CN111866533B (en) | Live broadcast transcoding method and device | |
| US7274739B2 (en) | Methods and apparatus for improving video quality in statistical multiplexing | |
| CN109831693B (en) | Method for reporting video frame information data and judging data underflow | |
| JP2007028598A (en) | Compression coding apparatus and compression coding method | |
| CN108924555B (en) | Code rate control bit distribution method suitable for video slice | |
| CN116962613A (en) | Data transmission method and device, computer equipment and storage medium | |
| KR101371507B1 (en) | System and method for low-delay video telecommunication | |
| JPH0744686B2 (en) | Variable transmission rate image coding device | |
| JP6300440B2 (en) | Image encoding device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |