WO2023071822A1 - 抗丢包方法、装置、电子设备及存储介质 - Google Patents

抗丢包方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023071822A1
WO2023071822A1 PCT/CN2022/125201 CN2022125201W WO2023071822A1 WO 2023071822 A1 WO2023071822 A1 WO 2023071822A1 CN 2022125201 W CN2022125201 W CN 2022125201W WO 2023071822 A1 WO2023071822 A1 WO 2023071822A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
information
channel redundancy
channel
packet loss
Prior art date
Application number
PCT/CN2022/125201
Other languages
English (en)
French (fr)
Inventor
刘丹
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2023071822A1 publication Critical patent/WO2023071822A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/167—Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0041—Arrangements at the transmitter end
    • H04L1/0042—Encoding specially adapted to other signal generation operation, e.g. in order to reduce transmit distortions, jitter, or to improve signal shape
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0056—Systems characterized by the type of code used
    • H04L1/007—Unequal error protection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/22—Arrangements for detecting or preventing errors in the information received using redundant apparatus to increase reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]

Definitions

  • the present invention relates to the communication field, in particular to an anti-packet loss method, device, electronic equipment and storage medium.
  • FEC Forward Error Correction
  • each frame of the code stream encoded by the vocoder will be in the form of [main frame + intra-frame redundancy], where the intra-frame redundancy information is to compare the important information of the previous frame or the previous frame with It is obtained by secondary encoding at a low code rate, and its length is only about 20% to 30% of the main frame. Therefore, whether it is the main frame or the intra-frame redundancy, the normal code stream can be decoded by the decoder alone.
  • the main frame will be called the first type of frame
  • the intra-frame redundancy will be called the second type. frame.
  • FEC Packet Loss Concealment, packet loss compensation technology
  • FEC cannot cope with continuous packet loss. When a certain frame is lost and future frames are also lost, it can only be compensated by the traditional PLC method. When the network packet loss rate is higher than 20%, there will be many continuous packet loss situations. At this time, FEC alone cannot ensure the intelligibility of speech.
  • the purpose of the present invention is to solve the above problems, provide an anti-packet loss method, device, electronic equipment and storage medium, realize the redundant information obtained by combining the FEC forward error correction function and channel redundancy, and according to the redundant information Restore the data of multiple frames in the past, thereby improving the packet loss compensation effect of the receiving end in the case of high packet loss rate.
  • an embodiment of the present application provides an anti-packet loss method, which is applied to the sending end, including: determining the channel redundancy configuration according to the current communication quality; wherein, the channel redundancy configuration is used to indicate the channel redundancy information Extract frame information; extract channel redundancy information according to the channel redundancy configuration and FEC forward error correction strategy, and send the channel redundancy information together with the code stream of the current frame to the receiving end for the receiving end to restore the lost frame based on the redundant information data; wherein, the redundant information includes channel redundant information and intra-frame redundant information in the code stream of the current frame.
  • the embodiment of the present application provides an anti-packet loss method, which is applied to the receiving end, including: obtaining the code stream and channel redundancy information of the current frame from the received data; In the case of frame data, the data of the lost frame is recovered according to the redundant information; wherein, the redundant information includes channel redundant information and intra-frame redundant information in the code stream of the current frame; wherein, the channel redundant information is determined by the sending end According to the channel redundancy configuration and the FEC forward error correction function extraction, the channel redundancy configuration is determined according to the current communication quality, and is used to indicate the extracted frame information of the channel redundancy information.
  • an embodiment of the present application provides an anti-packet loss device, which is applied to the sending end, including: a determination module, configured to determine the channel redundancy configuration according to the current communication quality; wherein, the channel redundancy configuration is used to indicate The extraction frame information of the channel redundancy information; the extraction module is used to extract the channel redundancy information according to the channel redundancy configuration and the FEC forward error correction strategy, and send the channel redundancy information to the receiving end together with the code stream of the current frame, The receiving end restores the data of the lost frame based on the redundancy information; wherein, the redundancy information includes channel redundancy information and intra-frame redundancy information in the code stream of the current frame.
  • an embodiment of the present application provides an anti-packet loss device, which is applied to the receiving end, including: an acquisition module, used to acquire the code stream and channel redundancy information of the current frame from the received data; The module is used to restore the data of the lost frame according to the redundant information when the redundant information includes the data of the lost frame; wherein, the redundant information includes channel redundant information and intra-frame redundant information in the code stream of the current frame;
  • the channel redundancy information is extracted by the sending end according to the channel redundancy configuration and the FEC forward error correction function, and the channel redundancy configuration is determined according to the current communication quality, and is used to indicate the extracted frame information of the channel redundancy information.
  • an embodiment of the present application also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores information that can be used by the Instructions executed by at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can execute the above-mentioned anti-packet loss method applied to the sending end, or can execute the above-mentioned anti-loss method applied to the receiving end package method.
  • an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program.
  • the computer program is executed by a processor, the above-mentioned anti-packet loss method applied to the sending end is implemented, or the above-mentioned method can be implemented.
  • Anti-packet loss method applied to the receiving end is implemented, or the above-mentioned method can be implemented.
  • the FEC strategy that is, the FEC function
  • the channel redundancy configuration is determined according to the communication quality, so that a smaller bandwidth can be occupied even in the case of a high packet loss rate
  • the receiving end restores the data of the lost frame according to the redundancy information in the above data packets, which obviously improves the packet loss compensation effect under the high packet loss rate.
  • FIG. 1 is a flowchart of an anti-packet loss method applied to a sending end provided by an embodiment of the present application
  • FIG. 2 is a flowchart of an anti-packet loss method applied to a receiving end provided by an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of an anti-packet loss device applied to a sending end provided by an embodiment of the present application
  • FIG. 4 is a schematic structural diagram of an anti-packet loss device applied to a receiving end provided by an embodiment of the present application
  • Fig. 5 is a schematic structural diagram of a communication device provided by an embodiment of the present application.
  • An embodiment of the present application relates to an anti-packet loss method, which is applied to the sending end, including: determining the channel redundancy configuration according to the current communication quality; wherein, the channel redundancy configuration is used to indicate the extracted frame information of the channel redundancy information; according to The channel redundancy configuration and the FEC forward error correction strategy extract the channel redundancy information, and send the channel redundancy information to the receiving end together with the code stream of the current frame, so that the receiving end can restore the data of the lost frame based on the redundant information; among them,
  • the redundancy information includes channel redundancy information and intra-frame redundancy information in the code stream of the current frame.
  • a channel redundancy configuration is determined according to the current communication quality, wherein the channel redundancy configuration is used to indicate extracted frame information of channel redundancy information.
  • the communication quality includes the network packet loss rate and bandwidth;
  • the channel redundancy configuration includes the channel redundancy frame number R, the frame number N of the first type frame in the channel redundancy information, and the number N of the first type frame in the channel redundancy information.
  • the frame number T and redundancy strategy of the second type of frame; wherein, R N+T, the sending end determines the R according to the network packet loss rate, and determines the proportion of N in R according to the bandwidth condition.
  • the sender determines the required number of channel redundancy frames R based on the code stream of the current frame, and ensures that the channel redundancy information carried by the increased R frame channel redundancy is the same as the current
  • the redundant information in the code stream of the frame can cover the past continuous R+1 frame data, that is, the channel redundancy information of 1 frame of the second type frame and R frame in the code stream of the current frame;
  • the sending end determines the The frame number N of the first type frame in the channel redundancy information (the value range of N is 0-R), since the channel redundancy is composed of the first type frame number and the second type frame number, then the second type frame number
  • the number of frames is R-N, wherein the first type of frame is the main frame in the code stream of each frame; the second type of frame is the intra-frame redundancy in the code stream of each frame.
  • the channel redundancy information can be dynamically changed according to the real-time network bandwidth situation.
  • the receiving end determines the number of channel redundant frames according to the current network packet loss rate, and determines the proportion of the number of frames of the first type of frame in the channel redundancy information in the number of channel redundant frames according to the current bandwidth situation, so that the sending end is in the channel Send more redundant information to the receiving end when the transmission can load, so that the receiving end can recover the data of the lost frame and improve the packet loss compensation effect.
  • the channel redundancy information consists of the n-2th frame (the first type frame), the n-3th frame (the Type II frame), n-4th frame (first type frame), n-5th frame (second type frame), n-6th frame (first type frame) and n-7th frame (second type frame) frame) composition. It should be noted that, the present application does not specifically limit the arrangements of the above-mentioned first-type frames and second-type frames.
  • first-type frames and second-type frames are cross-arranged in the channel redundancy information, so the first-type frames and the second-type frames in the channel redundancy information appear in pairs, because the vocoder with the FEC function
  • the code stream of the compiled frame is composed of the first type frame and the second type frame, therefore, the data of the extracted frame can be saved in the form of [first type frame + second type frame], so as not to The data of each frame needs to be saved separately, which better improves the effect of packet loss compensation.
  • the sending end before determining the channel redundancy configuration according to the communication quality, obtains the current communication quality, and the sending end obtains the code stream of the current frame, and stores the code stream of the current frame in the cache, wherein the current frame's
  • the code stream is encoded by the vocoder with the FEC function turned on, that is, the code stream of the current frame is composed of [first type frame + second type frame].
  • the bandwidth occupied by the first-type frames in the channel redundancy information is much larger than that of the second-type frames, the smaller N is, the less bandwidth is occupied, and the compensation effect will be worse.
  • the channel redundancy information is extracted according to the channel redundancy configuration and the FEC forward error correction strategy, and the channel redundancy information is sent to the receiving end together with the code stream of the current frame for the receiving end to recover the loss based on the redundancy information Data of the frame; wherein, the redundant information includes channel redundant information and intra-frame redundant information in the code stream of the current frame.
  • the sending end selects the code stream of the corresponding frame from the code streams of the stored frames, and extracts the information of the first type frame or the second type frame, and finally compares it with the current frame
  • the code streams are packaged together and sent to the receiving end.
  • the code stream compiled by the vocoder with the FEC function comes with the second type of frame information, combined with a reasonable channel plus redundancy method, it can effectively deal with continuous packet loss.
  • the frame number of the current frame is the nth frame
  • the FEC strategy is that the second type frame of the nth frame carries the information of the n-1th frame
  • the channel redundancy information is composed of the n-2th frame (the first type frame ), the n-3th frame (the second type of frame), then from the code stream of the cached n-2th frame, extract the first type of frame carrying the n-2th frame information and the n-3th frame
  • the second type of frame of frame information or from the code stream of the n-2th frame that has been cached, extract the first type of frame carrying the information of the n-2th frame, from the code stream of the n-1th frame that has been cached , the second type of frame carrying the n-3th frame information is extracted.
  • the first type of frames extracted above and the second form channel redundancy information. It should be noted that, in this application, the number of redundant frames in the channel redundancy information is not specifically limited.
  • the embodiment of the present application combines the FEC function with channel redundancy, and determines the channel redundancy configuration according to the current communication quality, so that the sender can send more redundant frames with less bandwidth occupation under high packet loss rate.
  • Data packets with redundant information the receiving end restores the data of the past multi-frames according to the redundant information, thereby improving the packet loss compensation effect of the receiving end in the case of high packet loss rate.
  • An embodiment of the present application relates to an anti-packet loss method, which is applied to the receiving end, including: obtaining the code stream and channel redundancy information of the current frame from the received data; when the redundant information includes the data of the lost frame Next, the data of the lost frame is restored according to the redundancy information; wherein, the redundancy information includes channel redundancy information and intra-frame redundancy information in the code stream of the current frame; wherein, the channel redundancy information is configured by the sender according to the channel redundancy and FEC forward error correction function extraction, the channel redundancy configuration is determined according to the current communication quality, and is used to indicate the extracted frame information of the channel redundancy information.
  • step 201 the code stream and channel redundancy information of the current frame are obtained from the received data.
  • the receiving end receives the data packet sent by the sending end, and extracts all the first-type frame and second-type frame information in the code stream of the current frame and channel redundancy information from the received data packet.
  • step 202 in the case that the redundant information contains the data of the lost frame, the data of the lost frame is restored according to the redundant information; wherein, the redundant information includes channel redundancy information and intra-frame redundancy in the code stream of the current frame information.
  • the receiving end lost the data of the n-3th frame, and the current data packet received by the receiving end is the code stream and channel redundancy information of the nth frame, where the channel redundancy information includes: n-2th frame (frames of the first type) and n-3 frames (frames of the second type). It can be known from the above that the channel redundancy information includes the data of the lost frame, so the receiving end can restore the data of the n-3th frame through the n-3th frame (second type frame) in the channel redundancy.
  • a traditional PLC algorithm is used to compensate the data.
  • the receiving end can effectively recover the data of the lost frame through the received redundant information, which improves the packet loss compensation effect.
  • this embodiment discloses an anti-packet loss method in a harsh network environment, which includes the following steps:
  • Step a1 The sender obtains the current network packet loss rate and bandwidth
  • Step a2 The sending end increases FEC redundancy and channel redundancy according to different grades of network packet loss rates, and sends them to the receiving end through RTP (Real-time Transport Protocol, Real-time Transport Protocol) protocol.
  • FEC redundancy refers to the proportion of redundant information in the code stream of the current frame.
  • the proportion of the above-mentioned redundant information will fluctuate within a preset range according to the network status. For example, if the current network status is poor, increase the redundancy The proportion of redundant information in the code stream of the current frame. If the current network status is good, reduce the proportion of redundant information in the code stream of the current frame.
  • the channel redundancy configuration is as follows:
  • the channel redundancy configuration is specifically as follows:
  • the channel redundancy information consists of the n-2th frame (the first type of frame) and the n-3th frame (the second type of frame); if the second type of frame carried by the code stream of the nth frame is the nth- 2 frames, then the channel redundancy information consists of the n-1th frame (the first type of frame) and the n-3th frame (the second type of frame).
  • the channel redundancy information added by this method contains 1 frame of the first type of frame and 1 frame of the second type of frame.
  • the channel redundancy information consists of the n-2th frame (the first type frame), the n-3th frame (the second type frame), the n-4th frame (the first type frame) and the n-5th frame (
  • the second type frame) is composed of; if the second type frame carried by the code stream of the nth frame is the n-2th frame, then the channel redundancy information is composed of the n-1th frame (the first type frame), the n-3th frame (the second type frame), the n-4th frame (the first type frame) and the n-6th frame (the second type frame).
  • the channel redundancy information added by this method includes 2 frames of the first type frame and 2 Frame The second type of frame.
  • the channel redundancy information consists of the n-2th frame (the first type frame), the n-3th frame (the second type frame), the n-4th frame (the first type frame), the n-5th frame ( The second type frame), the n-6th frame (the first type frame) and the n-7th frame (the second type frame); if the second type frame carried by the nth frame code stream is the n-2th frame, Then the channel redundancy information consists of the n-1th frame (the first type frame), the n-3th frame (the second type frame), the n-4th frame (the first type frame), the n-5th frame (the first type frame) class frame), n-6th frame (second class frame) and n-7th frame (second class frame).
  • the channel redundancy information consists of the n-1th frame (the first type frame), the n-3th frame (the second type frame), the n-4th frame (the first type frame), the n-5th frame (the first type frame) class frame), n-6th
  • the retransmission function needs to be turned on; if the bandwidth condition is not good, the retransmission function should be turned off.
  • Step a3 After receiving the data packet, the receiving end first puts it into the jitter buffer (ie JitterBuffer) for buffering, and then takes out and decodes the packet firstly stored in the JitterBuffer to obtain voice waveform data.
  • the jitter buffer ie JitterBuffer
  • the network packet loss rate is calculated as the average value within a period of time, it cannot be updated in time, which will cause the number of packets received by the receiving end to decrease sharply within a period of time. If the duration is long, the number of data packets cached in the JitterBuffer will be lower than the water level. At this time, it is necessary to stretch and decode the speech waveform data through the time scale change algorithm to reduce the consumption speed of the data packets in the JitterBuffer.
  • the receiving end will suddenly receive a large number of data packets in a short period of time, so that the number of data packets cached by the JitterBuffer is higher than the water level, then at this time, it is necessary to compress and decode through the time scale change algorithm to obtain voice waveform data to speed up the consumption of data packets in the JitterBuffer.
  • a retransmission request needs to be sent to the sender.
  • the sender has enabled the retransmission function, the retransmission command will be executed after receiving the retransmission request.
  • the receiving end finds that the sequence number of the next frame to be solved jumps, it needs to search for redundant information from the next packet to compensate for the lost voice frame. If no relevant redundant information is found, the PLC is enabled to compensate.
  • N and R in the above example and the arrangement of the first type frame and the second type frame in the channel redundancy are only for the purpose of specific description of the above embodiment, and the present application does not make any reference to N, R
  • the value of R and the arrangement of the first type frame and the second type frame in the channel redundancy are specifically limited.
  • a more severe network environment where the network packet loss rate is in the (40%, 100%] interval is disclosed. Due to the small bandwidth, a more bandwidth-saving anti-packet loss method is required.
  • the method Applicable to a multimedia communication engine includes the following steps:
  • Step b1 The sender obtains the current network packet loss rate and bandwidth
  • the channel redundancy information added by this method Contains 1 frame of the first type and 3 frames of the second type, the compensation effect is slightly worse, but it saves bandwidth.
  • the channel redundancy information added by this method includes 1 frame of the first type frame and 5 frames of the second type frame, and the compensation effect is slightly worse, but it saves more bandwidth.
  • Step b3 In the case of packet loss in the network, when the data needs to be restored, the receiving end needs to separate all the first type frame and the second type frame in the channel redundancy information from the data packet, and then from the above separated Look for information in the frame that can be used to recover the data.
  • an anti-packet loss method that minimizes bandwidth occupation and has little impact on compensation effects in a very harsh network environment is disclosed, which specifically includes the following steps:
  • Step c1 The sender obtains the current network packet loss rate and bandwidth
  • Step c2 Dynamically change the number of frames of a class of frames in the channel redundancy according to the bandwidth situation of the real-time network.
  • step c3 when there is a packet loss in the network, when the data needs to be restored, the receiving end needs to first separate all the first-type frames and the second-type frames in the channel redundancy frame from the data packet, and then separate from the above-mentioned Find out whether there is any information that can be used to recover the data in the outgoing frame.
  • step division of the above various methods is only for the sake of clarity of description. During implementation, it can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.
  • the embodiment of the present application also relates to an anti-packet loss device applied to the sending end, as shown in FIG. 3 , including: a determining module 301 and an extracting module 302 .
  • the determination module 301 is used to determine the channel redundancy configuration according to the current communication quality; wherein, the channel redundancy configuration is used to indicate the extracted frame information of the channel redundancy information; the extraction module 302 is used to determine the channel redundancy configuration according to the channel Redundancy configuration and FEC forward error correction strategy extract channel redundancy information, and send the channel redundancy information together with the code stream of the current frame to the receiving end for the receiving end to recover the data of the lost frame based on the redundant information ; Wherein, the redundancy information includes the channel redundancy information and the intra-frame redundancy information in the code stream of the current frame.
  • the communication quality includes the network packet loss rate and bandwidth;
  • the channel redundancy configuration includes the channel redundancy frame number R, the frame number N of the first type frame in the channel redundancy information, and the number N of the first type frame in the channel redundancy information.
  • the frame number T and redundancy strategy of the second type of frame; wherein, R N+T, the sending end determines the R according to the network packet loss rate, and determines the proportion of N in R according to the bandwidth condition.
  • the determination module 301 determines the required channel redundancy frame number R on the basis of the code stream of the current frame according to the current network packet loss rate, so as to ensure that the channel redundancy information carried by the increased R frame channel redundancy and The redundant information in the code stream of the current frame can cover the continuous R+1 frame data in the past, that is, the channel redundancy information of 1 frame of the second type frame and R frame in the code stream of the current frame; according to the bandwidth situation of the sending end, Determine the frame number N of the first type frame in the channel redundancy information (the value range of N is 0-R), since the channel redundancy is composed of the first type frame number and the second type frame, then the second type frame
  • the number of frames is R-N, wherein the first type of frame is the main frame in the code stream of each frame; the second type of frame is the intra-frame redundancy in the code stream of each frame.
  • the channel redundancy information can be dynamically changed according to the real-time network bandwidth situation.
  • the channel redundancy information consists of the n-2th frame (the first type frame), the n-3th frame (the Type II frame), n-4th frame (first type frame), n-5th frame (second type frame), n-6th frame (first type frame) and n-7th frame (second type frame) frame) composition. It should be noted that the present application does not specifically limit the arrangements of the above-mentioned first-type frames and second-type frames.
  • first-type frames and second-type frames are arranged in pairs, so that the first-type frames and second-type frames in the channel redundancy information appear in pairs, because the code stream of the frame coded by the vocoder with the FEC function turned on Both are composed of the first type frame and the second type frame, therefore, the data of the extracted frame can be saved in the form of [first type frame + second type frame], without the need for data of each frame It is saved separately to better improve the effect of packet loss compensation.
  • the sending end selects the code stream of the corresponding frame from the code streams of the stored frames according to the FEC strategy of the channel redundancy configuration, and extracts the information of the first type frame or the second type frame, and finally compares it with the current
  • the code stream of the frame is packaged together and sent to the receiving end.
  • the frame number of the current frame is the nth frame
  • the FEC strategy is that the second type frame of the nth frame carries the information of the n-1th frame
  • the channel redundancy information is determined by the n-2th frame (the first class frame), the n-3th frame (the second class frame), then from the code stream of the n-2th frame that has been cached, extract the first class frame carrying the information of the n-2th frame and the nth frame -3 frame information of the second type of frame; or from the code stream of the cached n-2th frame, extract the first type of frame carrying the n-2th frame information, from the cached n-1th frame In the code stream, the second type of frame carrying the n-3th frame information is extracted.
  • the first type of frames extracted above and the second form channel redundancy information. It should be noted that, in this application, the number of redundant frames in the channel redundancy information is not specifically limited.
  • the embodiment of the present application provides an anti-packet loss device applied to the sending end.
  • the sending end can Send data packets with more frames of redundant information with a smaller bandwidth occupation, and the receiving end restores the past multi-frame data according to the redundant information, thereby improving the packet loss compensation effect of the receiving end in the case of high packet loss rate .
  • this embodiment is a device embodiment corresponding to the above-mentioned embodiment of the anti-packet loss method applied to the sending end, and this embodiment can be implemented in cooperation with the above-mentioned embodiment of the anti-packet loss method applied to the sending end.
  • the relevant technical details mentioned above in the embodiment of the anti-packet loss method applied to the sending end are still valid in this embodiment, and will not be repeated here in order to reduce repetition.
  • the relevant technical details mentioned in this implementation manner can also be applied to the above embodiment of the anti-packet loss method at the application sending end.
  • the embodiment of the present application also relates to an anti-packet loss device applied to a receiving end, as shown in FIG. 4 , including: an acquisition module 401 and a recovery module 402 .
  • the obtaining module 401 is used to obtain the code stream and channel redundancy information of the current frame from the received data; Restoring the data of the lost frame with residual information; wherein, the redundant information includes the channel redundant information and the intra-frame redundant information in the code stream of the current frame; wherein the channel redundant information is determined by the sending end according to Channel redundancy configuration and FEC forward error correction function extraction, the channel redundancy configuration is determined according to the current communication quality, and is used to indicate the extracted frame information of channel redundancy information.
  • the receiving end lost the data of the n-3th frame, and the current data packet received by the receiving end is the code stream and channel redundancy information of the nth frame, where the channel redundancy information includes: n-2th frame (frames of the first type) and n-3 frames (frames of the second type). It can be seen from the above that the channel redundancy information includes the data of the lost frame, so the receiving end can restore the data of the n-3th frame through the n-3th frame (second type frame) in the channel redundancy.
  • a traditional PLC algorithm is used to compensate the data.
  • the embodiment of the present application provides an anti-packet loss device applied to the receiving end, which can effectively restore the data of the lost frame according to the received redundant information, and improve the effect of packet loss compensation.
  • this embodiment is a device embodiment corresponding to the above-mentioned embodiment of the anti-packet loss method applied to the receiving end, and this embodiment can be implemented in cooperation with the above-mentioned embodiment of the anti-packet loss method applied on the receiving end.
  • the relevant technical details mentioned above in the embodiment of the anti-packet loss method applied to the receiving end are still valid in this embodiment, and will not be repeated here to reduce repetition.
  • the relevant technical details mentioned in this implementation manner can also be applied to the above embodiment of the anti-packet loss method at the application receiving end.
  • modules involved in the above embodiments of the present application are logical modules.
  • a logical unit can be a physical unit, or a part of a physical unit, and can also Combination of physical units.
  • units that are not closely related to solving the technical problems proposed in the present application are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.
  • An embodiment of the present application also provides an electronic device, as shown in FIG. 5 , including at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; Instructions executed by the at least one processor 501, the instructions are executed by the at least one processor 501, so that the at least one processor can execute the above-mentioned anti-packet loss method applied to the sending end, or can execute the above-mentioned Anti-packet loss method applied to the receiving end.
  • the memory and the processor are connected by a bus
  • the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors and various circuits of the memory together.
  • the bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein.
  • the bus interface provides an interface between the bus and the transceivers.
  • a transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium.
  • the data processed by the processor is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor.
  • the processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Instead, memory can be used to store data that the processor uses when performing operations.
  • Embodiments of the present application also provide a computer-readable storage medium storing a computer program.
  • the above method embodiments are implemented when the computer program is executed by the processor.
  • a device which can be A single chip microcomputer, a chip, etc.
  • a processor processor
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)

Abstract

本申请公开了一种抗丢包方法、装置、电子设备及存储介质。方法包括：根据当前通信质量确定信道冗余配置；其中，信道冗余配置用于指示信道冗余信息的提取帧信息；根据信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将所述信道冗余信息与当前帧的码流一起发送至接收端，供接收端基于冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息。

Description

抗丢包方法、装置、电子设备及存储介质
本申请要求于2021年11月1日申请的、申请号为202111283639.1的中国专利申请的优先权，其全部内容通过引用结合在本申请中
技术领域
本发明涉及通信领域，尤其涉及一种抗丢包方法、装置、电子设备及存储介质。
背景技术
随着5G网络的发展，更大的网络带宽和更快的网速将使有FEC(Forward Error Correction，前向纠错)功能的声码器(比如SILK、OPUS、EVS等)成为语音通话的首选，以获得更好的音质和抗丢包性能。FEC功能指的是编码器在当前帧的码流中，嵌入将上一帧或者上上帧进一步压缩后的冗余信息，当某一帧丢失而未来帧可接收到的时候，解码器可以利用这些附加信息来恢复语音，以生成音质略微降低的补偿数据。启用FEC功能后，声码器编出的每一帧码流都会是[主帧+帧内冗余]的形式，其中帧内冗余信息是将上一帧或上上帧的重要信息以较低的码率进行二次编码得到的，它的长度大约只有主帧的20％～30％。因此，无论是主帧还是帧内冗余，单独给解码器都可以解出正常的码流，为了方便描述，后续将主帧称为第一类帧，将帧内冗余称为第二类帧。
与传统的PLC(Packet Loss Concealment，丢包补偿技术)算法相比，FEC在丢包时的语音可懂度和音质方面有明显优势。实际上，当网络丢包率在20％以下时，只用FEC就可以取得很好的丢包补偿效果。
然而，FEC无法应对连续丢包，当某一帧丢失而未来帧也丢掉的时候，只能按传统PLC的方法来补偿。在网络丢包率高于20％时，会出现很多连续丢包的情况，此时光靠FEC已经无法确保语音的可懂度。
发明内容
本发明的目的在于解决上述问题，提供一种抗丢包方法、装置、电子设备及存储介质，实现了通过结合FEC前向纠错功能和信道冗余得到冗余信息，并根据冗余信息来恢复过去多帧的数据，从而提高了接收端在高丢包率的情况下的丢包补偿效果。
为解决上述问题，本申请的实施例提供了一种抗丢包方法，应用于发送端，包括：根据当前通信质量确定信道冗余配置；其中，信道冗余配置用于指示信道冗余信息的提取帧信息；根据信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将信道冗余信息与当前帧的码流一起发送至接收端，供接收端基于冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息。
为解决上述问题，本申请的实施例提供了一种抗丢包方法，应用于接收端，包括：从接收的数据中，获取当前帧的码流和信道冗余信息；在冗余信息包含丢失帧的数据的情况下，根据冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息；其中，信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
为解决上述问题，本申请的实施例提供了一种抗丢包装置，应用于发送端，包括：确定模块，用于根据当前通信质量确定信道冗余配置；其中，信道冗余配置用于指示信道冗余信息的提取帧信息；提取模块，用于根据信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将信道冗余信息与当前帧的码流一起发送至接收端，供接收端基于冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息。
为解决上述问题，本申请的实施例提供了一种抗丢包装置，应用于接收端，包括：获取模块，用于从接收的数据中，获取当前帧的码流和信道冗余信息；恢复模块，用于冗余信息包含丢失帧的数据的情况下，根据冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息；其中，信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
为解决上述问题，本申请的实施例还提供了一种电子设备，包括：至少一个处理器；以及，与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行上述应用于发送端的抗丢包方法，或能够执行上述应用于接收端的抗丢包方法。
为解决上述问题，本申请的实施例还提供了一种计算机可读存储介质，存储有计算机程序，所述计算机程序被处理器执行时实现上述应用于发送端的抗丢包方法，或能够执行上述应用于接收端的抗丢包方法。
在本申请的实施例中，将FEC策略(即FEC功能)和信道冗余结合起来，并根据通信质量来确定信道冗余配置，使得在高丢包率情况下也能以较小的带宽占用来发送更多帧冗余信息的数据包，接收端根据上述数据包中的冗余信息来恢复丢失帧的数据，明显的提升了高丢包率下的丢包补偿效果。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请一实施例提供的应用于发送端的抗丢包方法的流程图；
图2是本申请一实施例提供的应用于接收端的抗丢包方法的流程图；
图3是本申请一实施例提供的应用于发送端的抗丢包装置的结构示意图；
图4是本申请一实施例提供的应用于接收端的抗丢包装置的结构示意图；
图5是本申请的一实施例提供的通信设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合附图对本申请的各实施方式进行详细的阐述。然而，本领域的普通技术人员可以理解，在本申请各实施方式中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施方式的种种变化和修改，也可以实现本申请所要求保护的技术方案。
本申请的一实施例涉及一种抗丢包方法，应用于发送端，包括：根据当前通信质量确定 信道冗余配置；其中，信道冗余配置用于指示信道冗余信息的提取帧信息；根据信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将信道冗余信息与当前帧的码流一起发送至接收端，供接收端基于冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息。
下面对本实施例中的抗丢包方法的实现细节进行具体的说明，以下内容仅为方便理解本方案的实现细节，并非实施本方案的必须。具体流程如图1所示，可包括如下步骤：
在步骤101中，根据当前通信质量确定信道冗余配置，其中，所述信道冗余配置用于指示信道冗余信息的提取帧信息。
在一个例子中，通信质量包括网络丢包率和带宽情况；信道冗余配置包括信道冗余帧数R、信道冗余信息中的第一类帧的帧数N、信道冗余信息中的第二类帧的帧数T和冗余策略；其中，R＝N+T，发送端根据网络丢包率确定所述R，根据带宽情况确定N在R中的占比。
在一个例子中，发送端根据当前网络丢包率，确定在当前帧的码流的基础上，需要的信道冗余帧数R，确保增加的R帧信道冗余携带的信道冗余信息和当前帧的码流中的冗余信息能覆盖过去连续的R+1帧数据，即当前帧的码流自带的1帧第二类帧和R帧信道冗余信息；发送端根据带宽情况，确定信道冗余信息中的第一类帧的帧数N(N的取值范围为0-R)，由于信道冗余由第一类帧数和第二类帧数构成，那么第二类帧的帧数就是R-N，其中，所述第一类帧是每一帧的码流中的主帧；所述第二类帧是每一帧的码流中的帧内冗余。
另外，由于网络丢包率统计的是一段时间内的均值，而带宽情况可以实时获取，所以，信道冗余帧数确定后，还可以根据实时网络的带宽情况动态地改变信道冗余信息中的第一类帧的帧数在信道冗余帧数中的占比，其中，带宽越大，信道冗余信息中的第一类帧的帧数在信道冗余帧数中的占比也越大。
接收端根据当前网络丢包率确定信道冗余帧数，根据当前的带宽情况确定信道冗余信息中的第一类帧的帧数在信道冗余帧数中的占比，使得发送端在信道传输能够负荷的情况下发送更多的冗余信息至接收端，供接收端恢复丢失的帧的数据，提高丢包补偿效果。
在一个例子中，冗余策略包括信道冗余信息中第一类帧和第二类帧的排列方式，排列方式包括：第一类帧和第二类帧交叉排列，例如：根据通信质量，确定信道冗余帧数R＝6，信道冗余信息中的第一类帧的帧数N＝3，那么第二类帧的帧数为3，如果当前帧的帧号为n，第n帧的码流携带的第二类帧为第n-1帧，根据冗余策略中的交叉排列方式，那么信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)、第n-5帧(第二类帧)、第n-6帧(第一类帧)和第n-7帧(第二类帧)组成。需要说明的是，本申请不对上述第一类帧和第二类帧的排列方式作具体的限定。
上述第一类帧和第二类帧在信道冗余信息中是交叉排列的方式，所以信道冗余信息中的第一类帧和第二类帧成对出现，由于打开FEC功能的声码器编出的帧的码流都是由第一类帧和第二类帧组成的，因此，可以将提取出的帧的数据以[第一类帧+第二类帧]的形式保存，从而不需要对每个帧的数据进行单独保存，更好的提高了丢包补偿效果。
在一个例子中，在根据通信质量确定信道冗余配置之前，发送端获取当前通信质量，以及发送端获取当前帧的码流，并将当前帧的码流存入缓存中，其中，当前帧的码流由打开FEC功能的声码器编出，即当前帧的码流由[第一类帧+第二类帧]的形式构成。
另外，由于信道冗余信息中的第一类帧占用的带宽远大于第二类帧，因此，N越小，占 用的带宽越少，补偿效果也就会越差，但即使N＝0，由于在当前帧的码流的基础上添加了冗余信息，所以补偿效果也会优于传统的OLC算法。
在步骤102中，根据信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将信道冗余信息与当前帧的码流一起发送至接收端，供接收端基于冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息。
具体地说，发送端根据信道冗余配置的FEC策略，从已存储的帧的码流中选择对应帧的码流，并提取其中的第一类帧或者第二类帧信息，最后与当前帧的码流一起打包发送至接收端。
打开FEC功能的声码器编出的码流自带第二类帧信息，再结合合理的信道加冗余的方法，可以有效应对连续的丢包。
在一个例子中，当前帧的帧号为第n帧，FEC策略为第n帧的第二类帧携带第n-1帧的信息，信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)组成，那么从已缓存的第n-2帧的码流中，提取出携带第n-2帧信息的第一类帧和携带第n-3帧信息的第二类帧；或者从已缓存的第n-2帧的码流中，提取出携带第n-2帧信息的第一类帧，从已缓存的第n-1帧的码流中，提取出携带第n-3帧信息的第二类帧。上述提取的第一类帧和第二组成信道冗余信息。需要说明的是，在本申请中，对信道冗余信息内的冗余帧数不作具体的限定。
本申请的实施例通过将FEC功能和信道冗余结合，并根据当前通信质量来确定信道冗余配置，从而使发送端能在高丢包率下以较小的带宽占用来发送更多帧冗余信息的数据包，接收端根据冗余信息来恢复过去多帧的数据，从而提高了接收端在高丢包率的情况下的丢包补偿效果。
本申请的一实施例涉及一种抗丢包方法，应用于接收端，包括：从接收的数据中，获取当前帧的码流和信道冗余信息；在冗余信息包含丢失帧的数据的情况下，根据冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息；其中，信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
下面对本实施例中的抗丢包方法的实现细节进行具体的说明，以下内容仅为方便理解本方案的实现细节，并非实施本方案的必须。具体流程如图2所示，可包括如下步骤：
在步骤201中，从接收的数据中，获取当前帧的码流和信道冗余信息。
在一个例子中，接收端接收发送端发送的数据包，从收到的数据包中拆出当前帧的码流和信道冗余信息中的所有第一类帧和第二类帧信息。
在步骤202中，在冗余信息包含丢失帧的数据的情况下，根据冗余信息恢复丢失帧的数据；其中，冗余信息包括信道冗余信息和当前帧的码流中的帧内冗余信息。
在一个例子中，接收端丢失了第n-3帧的数据，当下接收端收到的数据包是第n帧的码流和信道冗余信息，其中信道冗余信息包括：第n-2帧(第一类帧)和n-3帧(第二类帧)。由上述内容可知，信道冗余信息包含丢失帧的数据，所以接收端可以通过信道冗余中的第n-3帧(第二类帧)来恢复第n-3帧的数据。
在一个例子中，若收到的冗余信息中，没有丢失帧的数据，则采用传统的PLC算法来补偿数据。
在本申请实施例中，接收端通过收到的冗余信息，可以有效地恢复丢失帧的数据，提升 了丢包补偿效果。
为了使上述抗丢包方法过程更加清晰，本实施例公开一种恶劣网络环境下的抗丢包方法，该方法包括如下步骤：
步骤a1：发送端获取当前网络丢包率和带宽情况；
步骤a2：发送端根据不同档次的网络丢包率来增加FEC冗余和信道冗余，并通过RTP(Real-time Transport Protocol，实时传输协议)协议打包发送至接收端。其中，FEC冗余指的是当前帧的码流内冗余信息的占比，上述冗余信息的占比会根据网络状态在预设范围内浮动，例如，当前网络状态较差，则增加冗余信息在当前帧的码流内的占比，若当前网络状态良好，则减少冗余信息在当前帧的码流内的占比。
在带宽保持正常的情况下，发送端在面对不同档位网络丢包率时，信道冗余配置如下：
在一个例子中，如果网络丢包率在(0％，20％]区间，仅启用声码器的FEC功能，R＝0，N＝0，信道不加冗余；如果网络丢包率在(20％，40％]区间，R＝2，N＝1；如果网络丢包率在(40％，60％]区间，R＝4，N＝2；如果网络丢包率在(60％，100％]区间，R＝6，N＝3。信道冗余配置具体如下：
网络丢包率在(20％，40％]区间时，R＝2，N＝1，假设当前帧的帧号是n，如果第n帧的码流携带的第二类帧是第n-1帧，那么信道冗余信息由第n-2帧(第一类帧)和第n-3帧(第二类帧)组成；如果第n帧的码流携带的第二类帧是第n-2帧，那么信道冗余信息由第n-1帧(第一类帧)和第n-3帧(第二类帧)组成。这种方法加的信道冗余信息包含1帧第一类帧和1帧第二类帧。
网络丢包率在(40％，60％]区间时，R＝4，N＝2，假设当前帧的帧号是n，如果第n帧的码流携带的第二类帧是第n-1帧，那么信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)和第n-5帧(第二类帧)组成；如果第n帧的码流携带的第二类帧是第n-2帧，那么信道冗余信息由第n-1帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)和第n-6帧(第二类帧)组成。这种方法加的信道冗余信息包含2帧第一类帧和2帧第二类帧。
网络丢包率在(60％，100％]区间时，R＝6，N＝3，假设当前帧的帧号是n，如果第n帧的码流携带的第二类帧是第n-1帧，那么信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)、第n-5帧(第二类帧)、第n-6帧(第一类帧)和第n-7帧(第二类帧)组成；如果第n帧码流携带的第二类帧是第n-2帧，那么信道冗余信息由第n-1帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)、第n-5帧(第一类帧)、第n-6帧(第二类帧)和第n-7帧(第二类帧)组成。这种方法加的信道冗余信息包含3帧第一类帧和3帧第二类帧。
由上述内容可知，网络丢包率越大，发送端在当前帧的码流的基础上配置的信道冗余帧数越多，并且由于当前带宽情况保持正常，所以即使随着网络丢包率变大，也没有减少N在R中的占比。
另外，如果此时带宽情况良好，为了进一步确保丢包情况下的丢包补偿效果，还需打开重传功能：如果带宽情况不佳，则关闭重传功能。
步骤a3：接收端收到数据包后，先放入抖动缓冲器(即JitterBuffer)中缓存，再将最先存入JitterBuffer的包取出来解码，得到语音波形数据。
在一个例子中，如果网络中出现突发性丢包或者延时抖动，由于网络丢包率统计的是一段时间内的均值，不能及时更新，会使接收端在一段时间内收包个数锐减，持续时间长了就会使JitterBuffer中缓存的数据包个数低于水位。这个时候就需要通过时间尺度变化算法拉伸解码得到的语音波形数据，以降低JitterBuffer内数据包的消耗速度。如果网络中出现延时抖动，还会使接收端在短时间内突然收到大量的数据包，使JitterBuffer缓存的数据包个数高于水位，那么这个时候就需要通过时间尺度变化算法压缩解码得到的语音波形数据，以加快JitterBuffer内数据包的消耗速度。
如果JitterBuffer内的数据包低于水位或者马上要消耗完了，需向发送端发送重传请求。此时如果发送端打开了重传功能，收到重传请求后就会执行重传命令。
如果收到新的数据包与JitterBuffer中缓存的数据包序号不连续，证明网络发生了丢包或抖动。当接收端发现要解的下一帧序号跳变时，需从下一包中搜索冗余信息，以补偿丢失的语音帧。如果没有找到相关的冗余信息，则启用PLC来进行补偿。
另外，需要说明的是，上述例子中的N、R的值以及第一类帧和第二类帧在信道冗余中的排列方式只是为了对上述实施例作具体的说明，本申请不对N、R的值以及第一类帧和第二类帧在信道冗余中的排列方式作具体的限定。
在另一个例子中，公开了一种在网络丢包率在(40％，100％]区间的更加恶劣网络环境下，由于带宽小，所以需要一种更加节省带宽的抗丢包方法，该方法可应用于多媒体通讯引擎，该方法包括以下步骤：
步骤b1：发送端获取当前网络丢包率和带宽情况；
步骤b2：当网络丢包率在(40％，60％]区间时，信道冗余帧数R＝4，由于带宽小，因此，N＝1；当网络丢包率在(60％，100％]区间时，信道冗余帧数R＝6，N＝1，信道冗余具体如下：
网络丢包率在(40％，60％]区间时，R＝4，N＝1，假设当前帧的帧号是n，如果第n帧的码流携带的第二类帧是第n-1帧，那么组成信道冗余信息的第一类帧是第n-2帧；如果第n帧码流携带的第二类帧是第n-2帧，那么组成信道冗余信息的第一类帧是第n-1帧。无论哪种情况，组成信道冗余的第二类帧都是第n-3帧、第n-4帧和第n-5帧。这种方法加的信道冗余信息包含1帧第一类帧和3帧第二类帧，补偿效果稍差，但更节省带宽。
网络丢包率在(60％，100％]区间时，R＝6，N＝1，假设当前帧的帧号是n，如果第n帧码流携带的第二类帧是第n-1帧，那么组成信道冗余信息的第一类帧是第n-2帧；如果第n帧码流携带的第二类帧是第n-2帧，那么组成信道冗余信息的第一类帧是第n-1帧。无论哪种情况，组成信道冗余信息的第二类帧都是第n-3帧、第n-4帧、第n-5帧、第n-6帧和第n-7帧。这种方法加的信道冗余信息包含1帧第一类帧和5帧第二类帧，补偿效果稍差，但更节省带宽。
步骤b3：网络存在丢包的情况下，需要恢复数据时，接收端需先从数据包中将信道冗余信息中的第一类帧和第二类帧全部分离出来，再从上述分离出来的帧中寻找是否有能用来恢复数据的信息。
在另一个例子中，公开了一种在非常恶劣网络环境下，尽量减少带宽占用的同时，对补偿效果影响较小的抗丢包方法，具体包括如下步骤：
步骤c1：发送端获取当前网络丢包率和带宽情况；
步骤c2：根据实时网络的带宽情况动态改变信道冗余中的一类帧的帧数。
在一个例子中，丢包率高于60％的情况下，R＝6，信道冗余由N帧第一类帧和(6-N)第二类帧组成，其中N＝0～6，根据当前网络带宽情况来变化，N越小，占用带宽越少，补偿效果越差，但即使N＝0，补偿效果也会明显好于传统的PLC算法；
在步骤c3中，网络存在丢包的情况下，需要恢复数据时，接收端需先从数据包中将信道冗余帧中的第一类帧和第二类帧全部分离出来，再从上述分离出来的帧中寻找是否有能用来恢复数据的信息。
上述三个具体的例子是为了详细的描述本申请较佳的实施方式，但本发明的实施方式并不受上述实施例的限制，其他任何未背离本申请实施例的构思所做的改变、修饰、替代、组合、简化，都包含在本发明的保护范围之内。
上面各种方法的步骤划分，只是为了描述清楚，实现时可以合并为一个步骤或者对某些步骤进行拆分，分解为多个步骤，只要包括相同的逻辑关系，都在本专利的保护范围内；对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计，但不改变其算法和流程的核心设计都在该专利的保护范围内。
本申请实施例还涉及一种抗丢包装置、应用于发送端，如图3所示，包括：确定模块301和提取模块302。
具体地说，确定模块301，用于根据当前通信质量确定信道冗余配置；其中，所述信道冗余配置用于指示信道冗余信息的提取帧信息；提取模块302，用于根据所述信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将所述信道冗余信息与当前帧的码流一起发送至接收端，供所述接收端基于冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息。
在一个例子中，通信质量包括网络丢包率和带宽情况；信道冗余配置包括信道冗余帧数R、信道冗余信息中的第一类帧的帧数N、信道冗余信息中的第二类帧的帧数T和冗余策略；其中，R＝N+T，发送端根据网络丢包率确定所述R，根据带宽情况确定N在R中的占比。
在一个例子中，确定模块301根据当前网络丢包率，确定在当前帧的码流的基础上，需要的信道冗余帧数R，确保增加的R帧信道冗余携带的信道冗余信息和当前帧的码流中的冗余信息能覆盖过去连续的R+1帧数据，即当前帧的码流自带的1帧第二类帧和R帧信道冗余信息；发送端根据带宽情况，确定信道冗余信息中的第一类帧的帧数N(N的取值范围为0-R)，由于信道冗余由第一类帧数和第二类帧构成，那么第二类帧的帧数就是R-N，其中，所述第一类帧是每一帧的码流中的主帧；所述第二类帧是每一帧的码流中的帧内冗余。
另外，由于网络丢包率统计的是一段时间内的均值，而带宽情况可以实时获取，所以，信道冗余帧数确定后，还可以根据实时网络的带宽情况动态的改变信道冗余信息中的第一类帧的帧数在信道冗余帧数中的占比，其中，带宽越大，信道冗余信息中的第一类帧的帧数在信道冗余帧数中的占比也越大。
在一个例子中，冗余策略包括信道冗余信息中第一类帧和第二类帧的排列方式，排列方式包括：第一类帧和第二类帧交叉排列，例如：根据通信质量，确定信道冗余帧数R＝6，信道冗余信息中的第一类帧的帧数N＝3，那么第二类帧的帧数为3，如果当前帧的帧号为n，第n帧的码流携带的第二类帧为第n-1帧，根据冗余策略中的交叉排列方式，那么信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)、第n-4帧(第一类帧)、第n-5帧(第二类帧)、第n-6帧(第一类帧)和第n-7帧(第二类帧)组成。需要说明的是，本申请不对上述 第一类帧和第二类帧的排列方式作具体的限定。
上述第一类帧和第二类帧交叉排列的方式，使得信道冗余信息中的第一类帧和第二类帧成对出现，由于打开FEC功能的声码器编出的帧的码流都是由第一类帧和第二类帧组成的，因此，可以将提取出的帧的数据以[第一类帧+第二类帧]的形式保存，而不需要对每个帧的数据进行单独保存，从而更好的提高了丢包补偿效果。
在一个例子中，发送端根据信道冗余配置的FEC策略，从已存储的帧的码流中选择对应帧的码流，并提取其中的第一类帧或者第二类帧信息，最后与当前帧的码流一起打包发送至接收端。例如：在一个例子中，当前帧的帧号为第n帧，FEC策略为第n帧的第二类帧携带第n-1帧的信息，信道冗余信息由第n-2帧(第一类帧)、第n-3帧(第二类帧)组成，那么从已缓存的第n-2帧的码流中，提取出携带第n-2帧信息的第一类帧和携带第n-3帧信息的第二类帧；或者从已缓存的第n-2帧的码流中，提取出携带第n-2帧信息的第一类帧，从已缓存的第n-1帧的码流中，提取出携带第n-3帧信息的第二类帧。上述提取的第一类帧和第二组成信道冗余信息。需要说明的是，在本申请中，对信道冗余信息内的冗余帧数不作具体的限定。
本申请实施例提供的一种应用于发送端的抗丢包装置，通过将FEC功能和信道冗余结合，并根据当前通信质量来确定信道冗余配置，从而使发送端能在高丢包率下以较小的带宽占用来发送更多帧冗余信息的数据包，接收端根据冗余信息来恢复过去多帧的数据，从而提高了接收端在高丢包率的情况下的丢包补偿效果。
不难发现，本实施方式为上述应用于发送端的抗丢包方法实施例相对应的装置实施例，本实施方式可与上述应用于发送端的抗丢包方法实施例互相配合实施。上述应用于发送端的抗丢包方法实施例提到的相关技术细节在本实施方式中依然有效，为了减少重复，这里不再赘述。相应地，本实施方式中提到的相关技术细节也可应用在上述应用发送端的抗丢包方法实施例中。
本申请实施例还涉及一种抗丢包装置、应用于接收端，如图4所示，包括：获取模块401和恢复模块402。
具体地说，获取模块401，用于从接收的数据中，获取当前帧的码流和信道冗余信息；恢复模块402，用于冗余信息包含丢失帧的数据的情况下，根据所述冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息；其中，所述信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，所述信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
在一个例子中，接收端丢失了第n-3帧的数据，当下接收端收到的数据包是第n帧的码流和信道冗余信息，其中信道冗余信息包括：第n-2帧(第一类帧)和n-3帧(第二类帧)。有上述内容可知，信道冗余信息包含丢失帧的数据，所以接收端可以通过信道冗余中的第n-3帧(第二类帧)来恢复第n-3帧的数据。
在一个例子中，若收到的冗余信息中，没有丢失帧的信息，则采用传统的PLC算法来补偿数据。
本申请实施例提供的一种应用于接收端的抗丢包装置，根据收到的冗余信息，可以有效的恢复丢失帧的数据，提升了丢包补偿效果。
不难发现，本实施方式为上述应用于接收端的抗丢包方法实施例相对应的装置实施例， 本实施方式可与上述应用于接收端的抗丢包方法实施例互相配合实施。上述应用于接收端的抗丢包方法实施例提到的相关技术细节在本实施方式中依然有效，为了减少重复，这里不再赘述。相应地，本实施方式中提到的相关技术细节也可应用在上述应用接收端的抗丢包方法实施例中。
值得一提的是，本申请上述实施方式中所涉及到的各模块均为逻辑模块，在实际应用中，一个逻辑单元可以是一个物理单元，也可以是一个物理单元的一部分，还可以以多个物理单元的组合实现。此外，为了突出本申请的创新部分，本实施方式中并没有将与解决本申请所提出的技术问题关系不太密切的单元引入，但这并不表明本实施方式中不存在其它的单元。
本申请的实施例还提供一种电子设备，如图5所示，包括至少一个处理器501；以及，与所述至少一个处理器501通信连接的存储器502；其中，所述存储器502存储有可被所述至少一个处理器501执行的指令，所述指令被所述至少一个处理器501执行，以使所述至少一个处理器能够执行上述应用于发送端的抗丢包方法，或者，能够执行上述应用于接收端的抗丢包方法。
其中，存储器和处理器采用总线方式连接，总线可以包括任意数量的互联的总线和桥，总线将一个或多个处理器和存储器的各种电路连接在一起。总线还可以将诸如***设备、稳压器和功率管理电路等之类的各种其他电路连接在一起，这些都是本领域所公知的，因此，本文不再对其进行进一步描述。总线接口在总线和收发机之间提供接口。收发机可以是一个元件，也可以是多个元件，比如多个接收器和发送器，提供用于在传输介质上与各种其他装置通信的单元。经处理器处理的数据通过天线在无线介质上进行传输，进一步，天线还接收数据并将数据传送给处理器。
处理器负责管理总线和通常的处理，还可以提供各种功能，包括定时，***接口，电压调节、电源管理以及其他控制功能。而存储器可以被用于存储处理器在执行操作时所使用的数据。
上述产品可执行本申请实施例所提供的方法，具备执行方法相应的功能模块和有益效果，未在本实施例中详尽描述的技术细节，可参见本申请实施例所提供的方法。
本申请的实施例还提供一种计算机可读存储介质，存储有计算机程序。计算机程序被处理器执行时实现上述方法实施例。
本领域技术人员可以理解，实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
上述实施例是提供给本领域普通技术人员来实现和使用本申请的，本领域普通技术人员可以在脱离本申请的发明思想的情况下，对上述实施例做出种种修改或变化，因而本申请的保护范围并不被上述实施例所限，而应该符合权利要求书所提到的创新性特征的最大范围。

Claims (11)

  1. 一种抗丢包方法，其中，应用于发送端，包括：
    根据当前通信质量确定信道冗余配置；其中，所述信道冗余配置用于指示信道冗余信息的提取帧信息；
    根据所述信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将所述信道冗余信息与当前帧的码流一起发送至接收端，供所述接收端基于冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息。
  2. 根据权利要求1所述的抗丢包方法，其中，所述信道冗余配置，包括：
    信道冗余帧数R、所述信道冗余信息中的第一类帧的帧数N、所述信道冗余信息中的第二类帧的帧数T和冗余策略；其中，R＝N+T；
    其中，所述第一类帧是每一帧的码流中的主帧；所述第二类帧是每一帧的码流中的帧内冗余；所述冗余策略包括所述信道冗余信息中的所述第一类帧和所述第二类帧的排列方式。
  3. 根据权利要求2所述的抗丢包方法，其中，所述排列方式包括：所述第一类帧和所述第二类帧交叉排列。
  4. 根据权利要求2所述的抗丢包方法，其中，所述通信质量包括：网络丢包率和带宽情况；
    其中，所述网络丢包率用于确定所述R，所述带宽情况用于确定所述N在所述R中的占比。
  5. 根据权利要求1至4中任一项所述的抗丢包方法，其中，所述根据所述信道冗余配置和FEC前向纠错策略提取信道冗余信息，包括：
    根据所述信道冗余配置和FEC前向纠错策略，从已缓存的帧的码流中提取所述信道冗余信息。
  6. 一种抗丢包方法，其中，应用于接收端，包括：
    从接收的数据中，获取当前帧的码流和信道冗余信息；
    在冗余信息包含丢失帧的数据的情况下，根据所述冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息；
    其中，所述信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，所述信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
  7. 根据权利要求6所述的抗丢包方法，其中，所述通信质量包括：网路丢包率和带宽情况；
    其中，所述网络丢包率用于确定信道冗余帧数R，所述带宽情况用于确定信道冗余信息中的第一类帧的帧数N在所述R中的占比。
  8. 一种抗丢包装置，其中，应用于发送端，包括：
    确定模块，用于根据当前通信质量确定信道冗余配置；其中，所述信道冗余配置用于指示信道冗余信息的提取帧信息；
    提取模块，用于根据所述信道冗余配置和FEC前向纠错策略提取信道冗余信息，并将所述信道冗余信息与当前帧的码流一起发送至接收端，供所述接收端基于冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息。
  9. 一种抗丢包装置，其中，应用于接收端，包括：
    获取模块，用于从接收的数据中，获取当前帧的码流和信道冗余信息；
    恢复模块，用于冗余信息包含丢失帧的数据的情况下，根据所述冗余信息恢复丢失帧的数据；其中，所述冗余信息包括所述信道冗余信息和所述当前帧的码流中的帧内冗余信息；
    其中，所述信道冗余信息由发送端根据信道冗余配置和FEC前向纠错功能提取，所述信道冗余配置根据当前通信质量确定，用于指示信道冗余信息的提取帧信息。
  10. 一种电子设备，其中，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如权利要求1至5中任一项所述的抗丢包方法，或者能够执行如权利要求6-7任一项所述的抗丢包方法。
  11. 一种计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至5中任一项所述的抗丢包方法，或者能够执行如权利要求6-7任一项所述的抗丢包方法。
PCT/CN2022/125201 2021-11-01 2022-10-13 抗丢包方法、装置、电子设备及存储介质 WO2023071822A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111283639.1A CN116073946A (zh) 2021-11-01 2021-11-01 抗丢包方法、装置、电子设备及存储介质
CN202111283639.1 2021-11-01

Publications (1)

Publication Number Publication Date
WO2023071822A1 true WO2023071822A1 (zh) 2023-05-04

Family

ID=86159120

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/125201 WO2023071822A1 (zh) 2021-11-01 2022-10-13 抗丢包方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN116073946A (zh)
WO (1) WO2023071822A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116847453B (zh) * 2023-09-04 2023-11-14 四川轻化工大学 卫星数据传输及卫星物联网接入时间的管理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150006991A1 (en) * 2012-01-25 2015-01-01 Electronics And Telecommunications Research Institute Graceful degradation-forward error correction method and apparatus for performing same
CN105161114A (zh) * 2011-04-11 2015-12-16 三星电子株式会社 用于多码率语音和音频编解码器的帧擦除隐藏
CN105933342A (zh) * 2016-06-24 2016-09-07 浙江宇视科技有限公司 一种视频监控数据的传输方法及装置
CN110890945A (zh) * 2019-11-20 2020-03-17 腾讯科技（深圳）有限公司 数据传输方法、装置、终端及存储介质
CN112820306A (zh) * 2020-02-20 2021-05-18 腾讯科技（深圳）有限公司 语音传输方法、***、装置、计算机可读存储介质和设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105161114A (zh) * 2011-04-11 2015-12-16 三星电子株式会社 用于多码率语音和音频编解码器的帧擦除隐藏
US20150006991A1 (en) * 2012-01-25 2015-01-01 Electronics And Telecommunications Research Institute Graceful degradation-forward error correction method and apparatus for performing same
CN105933342A (zh) * 2016-06-24 2016-09-07 浙江宇视科技有限公司 一种视频监控数据的传输方法及装置
CN110890945A (zh) * 2019-11-20 2020-03-17 腾讯科技（深圳）有限公司 数据传输方法、装置、终端及存储介质
CN112820306A (zh) * 2020-02-20 2021-05-18 腾讯科技（深圳）有限公司 语音传输方法、***、装置、计算机可读存储介质和设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
J. UBERTI: "WebRTC Forward Error Correction Requirements", IETF RFC8854, 8854, 19 January 2021 (2021-01-19), XP015143846 *

Also Published As

Publication number Publication date
CN116073946A (zh) 2023-05-05

Similar Documents

Publication Publication Date Title
US11227612B2 (en) Audio frame loss and recovery with redundant frames
CN102984090B (zh) 一种用于发送实时通信事件数据到接收器的抖动缓冲器的发送器及其方法
WO2017157303A1 (zh) 实时通信中的抗丢包方法、装置和***
US9246644B2 (en) Jitter buffer
CN102984091B (zh) 抖动缓冲器
WO2023071822A1 (zh) 抗丢包方法、装置、电子设备及存储介质
CN101051465B (zh) 用于解码被编码的有用数据的方法和解码装置
CN102130886A (zh) 网络视频流媒体***及传输处理方法、发送端和接收端
CN113242155A (zh) 一种数据包丢包的恢复方法、***及计算机可读存储介质
WO2016142193A1 (en) Apparatus and method for transmitting and receiving redundancy information of a first packet in a second packet
CN101192903B (zh) 数据帧编解码控制方法
CN103050123B (zh) 一种传输语音信息的方法和***
WO2011012029A1 (zh) 多描述音频编解码的方法、装置及***
CN112769524B (zh) 语音传输方法、装置、计算机设备和存储介质
WO2002017588A1 (fr) Procede de codage de la parole a basse vitesse sur la base d'un protocole reseau
CN102547284A (zh) 便携式无线高清图像传输***及其方法
CN114286132A (zh) 一种无线网络中视频传输容错的方法
AU2012100851A4 (en) Multiple description video codec based on adaptive temporal sub-sampling
CN113438054B (zh) 数据传输方法及装置
CN115720130A (zh) 音频处理方法、装置、终端及计算机可读存储介质
Tu et al. A research on adaptive QoS technology
CN202385225U (zh) 便携式无线高清图像传输***
CN107835422B (zh) 一种基于显著性的hevc多描述图像编码算法
CN117014697A (zh) 基于喷泉码的自适应视频传输方法、电子设备和存储介质
CN113938346A (zh) Mvb端口数据处理方法、mvb通信网络及轨道交通车辆

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22885700

Country of ref document: EP

Kind code of ref document: A1