CN112929667A - Encoding and decoding method, device and equipment and readable storage medium - Google Patents

Encoding and decoding method, device and equipment and readable storage medium Download PDF

Info

Publication number
CN112929667A
CN112929667A CN202110325525.2A CN202110325525A CN112929667A CN 112929667 A CN112929667 A CN 112929667A CN 202110325525 A CN202110325525 A CN 202110325525A CN 112929667 A CN112929667 A CN 112929667A
Authority
CN
China
Prior art keywords
image
frame
determining
intra
frames
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110325525.2A
Other languages
Chinese (zh)
Other versions
CN112929667B (en
Inventor
张志辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Migu Cultural Technology Co Ltd
China Mobile Communications Group Co Ltd
Original Assignee
Migu Cultural Technology Co Ltd
China Mobile Communications Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Migu Cultural Technology Co Ltd, China Mobile Communications Group Co Ltd filed Critical Migu Cultural Technology Co Ltd
Priority to CN202110325525.2A priority Critical patent/CN112929667B/en
Publication of CN112929667A publication Critical patent/CN112929667A/en
Application granted granted Critical
Publication of CN112929667B publication Critical patent/CN112929667B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention provides a coding and decoding method, a device, equipment and a readable storage medium, wherein the coding method comprises the following steps: determining at least two consecutive image frames from a current group of pictures, GOP, in response to a received instantaneous decoding refresh, IDR, request; determining a first partial image area from the respective image areas of the at least two consecutive image frames as the respective intra-coded image areas of the at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame; in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode. The scheme well solves the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction.

Description

Encoding and decoding method, device and equipment and readable storage medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a coding and decoding method, apparatus, device, and readable storage medium.
Background
In the prior art, when a network has a packet loss and an error code (that is, a video has a screen splash phenomenon), an IDR (Instantaneous Decoding Refresh) Refresh request is sent to a sending end through a receiving end, and after the sending end receives the IDR request sent by the receiving end, the sending end immediately encodes a video sequence using an IDR video frame as an initial frame and sends the video sequence to the receiving end, so that the screen splash phenomenon can disappear after the IDR video frame is completely received and decoded by the receiving end.
However, in the prior art, the code rate is controlled and the video sequence is transmitted according to a Group of Pictures (GOP), and the bandwidth occupied by the IDR frame is generally 10 times of the bandwidth occupied by the P frame (forward predictive coding frame), so if a new IDR frame is inserted suddenly at present, the code rate may suddenly increase, which causes network jitter, packet loss, congestion, and stutter, but cannot solve the problem of screen splash, and also introduces the problems of sudden increase of network packet loss, sudden decrease of image quality, and the like, which causes the following video to be stuttered more and more decoding failures.
As can be seen from the above, the conventional codec scheme for IDR request has the problem of easily causing image quality degradation.
Disclosure of Invention
The invention aims to provide a coding and decoding method, a coding and decoding device, coding and decoding equipment and a readable storage medium, which aim to solve the problem that the coding and decoding scheme aiming at an IDR request in the prior art is easy to cause image quality reduction.
In order to solve the above technical problem, an embodiment of the present invention provides an encoding method, including:
determining at least two consecutive image frames from a current group of pictures, GOP, in response to a received instantaneous decoding refresh, IDR, request;
determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame;
in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
Optionally, the method further includes:
respectively determining second local image areas from respective image areas of a specified image frame as respective inter-frame coding image areas of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area;
in encoding the specified image frame, performing: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
Optionally, in response to the received instantaneous decoding refresh IDR request, determining at least two consecutive image frames from the current group of pictures GOP, including:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
Optionally, determining an index value characterizing network quality in response to the IDR request includes:
responding to the IDR request, and judging whether the current image frame meets the judgment condition of the limited image frame;
and if so, determining the index value.
Optionally, the determining whether the current image frame meets the determination condition of the limited image frame includes:
and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
Optionally, determining whether the current image frame to be encoded meets the determination condition for the limited image frame according to the arrangement position of the current image frame in the current GOP includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back;
if yes, determining that the current image frame meets the limited image frame judgment condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
Optionally, the index value includes a round-trip delay rtt value;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
Optionally, the determining, from the image areas of the at least two consecutive image frames, the first local image area as the intra-frame coding image area of the at least two consecutive image frames respectively includes:
and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
The embodiment of the invention also provides a decoding method, which comprises the following steps:
sending an immediate decoding refresh IDR request;
receiving an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
An embodiment of the present invention further provides an encoding apparatus, including:
a first determining module for determining at least two consecutive image frames from a current group of pictures, GOP, in response to a received instantaneous decoding refresh, IDR, request;
a second determining module, configured to determine, from respective image areas of the at least two consecutive image frames, a first local image area as an intra-coded image area of the at least two consecutive image frames, respectively; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame;
a first encoding module for performing, when encoding the at least two consecutive image frames: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
Optionally, the apparatus further comprises:
a third determining module, configured to determine second local image regions from respective image regions of a specified image frame, as inter-frame encoded image regions of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area;
a second encoding module, configured to, when encoding the specified image frame, perform: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
Optionally, in response to the received instantaneous decoding refresh IDR request, determining at least two consecutive image frames from the current group of pictures GOP, including:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
Optionally, determining an index value characterizing network quality in response to the IDR request includes:
responding to the IDR request, and judging whether the current image frame meets the judgment condition of the limited image frame;
and if so, determining the index value.
Optionally, the determining whether the current image frame meets the determination condition of the limited image frame includes:
and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
Optionally, determining whether the current image frame to be encoded meets the determination condition for the limited image frame according to the arrangement position of the current image frame in the current GOP includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back;
if yes, determining that the current image frame meets the limited image frame judgment condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
Optionally, the index value includes a round-trip delay rtt value;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
Optionally, the determining, from the image areas of the at least two consecutive image frames, the first local image area as the intra-frame coding image area of the at least two consecutive image frames respectively includes:
and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
An embodiment of the present invention further provides a decoding apparatus, including:
the first sending module is used for sending an instantaneous decoding and refreshing IDR request;
a first receiving module, configured to receive an image frame obtained by an encoding end encoding at least two consecutive image frames in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
The embodiment of the invention also provides coding equipment, which comprises a memory, a processor and a program which is stored on the memory and can run on the processor; the processor implements the above-described encoding method when executing the program.
The embodiment of the invention also provides decoding equipment, which comprises a memory, a processor and a program which is stored on the memory and can be operated on the processor; the processor implements the above decoding method when executing the program.
The embodiment of the invention also provides a readable storage medium, wherein a program is stored on the readable storage medium, and the program realizes the steps in the coding method when being executed by a processor; alternatively, the program realizes the steps in the above decoding method when executed by a processor.
The technical scheme of the invention has the following beneficial effects:
in the above solution, the encoding method determines at least two consecutive image frames from the current group of pictures GOP by responding to the received instantaneous decoding refresh IDR request; determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame; in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode. By the scheme of the embodiment of the application, after the IDR request is received, intra-frame coding can be performed on the local image areas of the image frames to achieve the effect of a complete IDR frame, instead of directly coding at least one image frame into the complete IDR frame to be transmitted, so that an IDR frame with a larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with a smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, the problems of more packet loss, network jitter, image quality reduction and the like caused by the coding and decoding scheme aiming at the IDR request in the prior art are solved well, and the problem of image quality reduction caused by the coding and decoding scheme aiming at the IDR request is solved well.
Drawings
FIG. 1 is a flow chart of an encoding method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of coverage of the entire image area according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a decoding method according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a frame for implementing an encoding method according to an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an encoding apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a decoding device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages of the present invention more apparent, the following detailed description is given with reference to the accompanying drawings and specific embodiments.
Aiming at the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction, the invention provides an encoding method, as shown in figure 1, which comprises the following steps:
step 11: at least two consecutive image frames are determined from the current group of pictures, GOP, in response to the received instantaneous decoding refresh, IDR, request.
The current GOP refers to a GOP to which a first image frame to be encoded belongs when the IDR request is received. The image frame of the determined "at least two consecutive image frames" may include a frame before the first image frame, and/or a frame after the first image frame, which is not limited herein.
Step 12: determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in the corresponding image areas, and can collectively cover the entire image area of a single image frame.
Wherein "collectively cover the entire image area of a single said image frame" can be understood as: an entire image area corresponding to one image frame; the entire image area of different image frames may be the same; the common coverage may be the common coverage with or without overlap of image areas. Taking the case of common coverage without overlapping as an example, assuming that two consecutive frames, i.e., an a frame and a B frame, are selected, as shown in fig. 2, an intra-coded image area a of the a frame and an intra-coded image area B of the B frame can collectively cover the entire image area c of one image frame.
Step 13: in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
In this step, the intra-frame coding part of the IDR frame fed back after the IDR request is split into the intra-frame coding parts of a plurality of continuous frames for coding respectively, so that one IDR frame occupying a larger bandwidth is split into a plurality of local intra-frame coding frames occupying a smaller bandwidth, thereby reducing the requirement on the bandwidth and avoiding the situations of packet loss, jamming and the like.
In view of the above, the encoding method provided by the embodiment of the present invention determines at least two consecutive image frames from a current group of pictures GOP by responding to a received instantaneous decoding refresh IDR request; determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame; in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode. By the scheme of the embodiment of the application, after receiving the IDR request, intra-frame coding can be performed on the local image areas of the image frames to achieve the effect of a complete IDR frame, rather than directly coding at least one image frame into the complete IDR frame for transmission, so that one IDR frame with a larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with a smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction is well solved.
Further, the method further comprises: respectively determining second local image areas from respective image areas of a specified image frame as respective inter-frame coding image areas of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area; in encoding the specified image frame, performing: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
Thus, since there is an intra-frame coded frame that can be referred to forward from the above-mentioned determination of the first frame of at least two consecutive image frames (i.e., the first frame of limited coding), it is possible to perform coding using a local forward prediction coding method starting from the second frame. In the above process, the inter-coded picture region generally corresponds to the picture region of the previous frame to which the inter-coded picture region refers, but the invention is not limited thereto.
In the implementation of the present invention, further, the inter-frame coding image region may be coded with reference to the first partial image region and the second partial image region in the previous image frame.
Specifically, in this embodiment of the present invention, the operation of determining the first local image area and the second local image area may include:
determining a limited area (including a first local area image area and a second local area image area) of each of the at least two consecutive image frames; the frame N +1 is one more limited area than the frame N, and N is an integer greater than or equal to 0;
and encoding the last limited area (namely, the first local image area) in the limited area corresponding to each image frame in an intra-frame encoding mode, and encoding the residual limited area (namely, the second local image area) in an inter-frame encoding mode under the condition that the residual limited area exists, wherein the inter-frame encoding refers to all limited areas in the previous image frame.
The number of the limited regions of the first image frame (i.e., the current image frame) in at least two consecutive image frames may be 1, the number of the limited regions of the next image frame after the first image frame may be 2, and so on, and the number of the limited regions of the subsequent image frames may be sequentially increased; the "previous image frame" refers to a frame previous to the present image frame (the image frame corresponding to the remaining limited region).
In this way, the scheme can realize block refreshing (intra-frame coding), and the coding mode can ensure that the coded data can be normally displayed; in addition, the intra-frame coding mode can ensure that data refreshing is realized and subsequent normal display is ensured; and the interframe coding can ensure that the reference is the refreshed data, thereby ensuring the subsequent normal display.
In the embodiment of the present invention, whether to adopt the method shown in fig. 1 may be determined according to the network quality. This is because the quality of the network determines to some extent whether the conventional IDR coding has the problems as described in the background art.
In view of the above, in one embodiment, determining at least two consecutive image frames from the current group of pictures GOP in response to the received instantaneous decoding refresh IDR request may include:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
Specifically, before determining the index value, it may be determined whether the current image frame meets a limited image frame determination condition (to be described later) in response to the IDR request. And if so, determining the index value. The index value may include a value of rtt (round-trip time).
In the embodiment of the invention, under the condition of relatively good network quality, the rtt value is relatively small, and a relatively small number of image frames needing limited processing can be determined according to the rtt value; under the condition that the network quality is relatively poor, the rtt value is relatively large, and a relatively large number of image frames needing limited processing can be determined according to the rtt value; therefore, the frame refreshing is realized as well as possible no matter the network quality; that is, the image frames requiring limited processing can be determined according to the network quality, so that the frame refreshing can be better realized, and the problems of jamming and the like can be avoided.
Here, the "determining the index value" in the current image frame, i.e., the first image frame referred to above may also be understood as obtaining the index value.
In the embodiment of the present invention, the determining whether the current image frame meets the determination condition of the limited image frame includes: and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
In the embodiment of the present invention, determining whether the current image frame to be encoded meets the determination condition for the limited image frame according to the arrangement position of the current image frame in the current GOP includes: judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back; if yes, determining that the current image frame meets the limited image frame judgment condition; if not, determining that the current image frame does not meet the limited image judgment condition.
Whether relatively forward or relatively backward refers to whether the current image frame is relatively forward or backward in position in the entire current GOP. For example, the current image frame is positioned at the forefront in the entire current GOP, and is considered to be relatively advanced; alternatively, the current image frame may be considered relatively late if its position in the entire current GOP is last.
The arrangement position can be represented by a position ratio, namely whether the judgment condition is met or not is determined according to the position condition of the current image frame in the current GOP. The position ratio refers to: the ratio between the position ordering value (i.e. number M) of the current image frame in the current GOP and (the time length T and the frame rate fps corresponding to the GOP).
Specifically, it may be determined whether the position occupation ratio corresponding to the arrangement position is within a preset value range, where the preset value range is 0 to 0.2, or 0.8 to 1. That is, it is judged whether the arrangement position is in the first 20% or the last 20% of the entire GOP. The special processing is performed on the previous frame or the next frame, because the previous (or next) frame is closer to the IDR frame of the previous (or next) frame group, which occupies a larger bandwidth and causes the seizure, so that the occupied bandwidth can be reduced to a certain extent by the special processing, and the seizure is avoided. It can also be understood that: in the embodiment of the present invention, it is considered that limited coding is performed by taking 20% of the previous frame and the next frame in the GOP, because if too far ahead frame is taken as IDR, the following problem occurs: just sending one IDR and sending another IDR can occupy too much bandwidth. Similarly, if a frame too late is considered an IDR, the following problem occurs: the latter IDR is affected. Therefore, the two frames need to be selected for limited encoding.
In the embodiment of the present invention, the absolute value of the difference between the two end values of the preset value range is multiplied by the total number of frames corresponding to the GOP, and is greater than or equal to 3 frames, but not limited thereto.
In the embodiment of the present invention, the index value includes a value of rtt (round-trip time); determining the at least two consecutive image frames from the current GOP according to the index value, including: determining the size of a target time window according to the rtt value and the rtt threshold value; and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
This can control the effect of implementing I-frame refresh with a minimum of frames. The rtt threshold is greater than or equal to 100ms and less than or equal to 300ms, and may be 200 ms.
Specifically, when the rtt value is less than or equal to the rtt threshold, a first formula is adopted to determine the size of the target time window (specifically, the size may be the number of corresponding frames); under the condition that the rtt value is larger than the rtt threshold value, determining the size of the target time window by adopting a second formula; wherein the first formula is: w ═ min (3, max (500/rtt, 5)); the second formula is: w ═ max (3,500/rtt); w represents the size of the target time window and rtt represents the value of rtt. The first formula represents taking the maximum of 500/rtt and 5, and then taking the minimum of the maximum and 3; the second formula represents taking the maximum of 3 and 500/rtt.
Thus if rtt ≦ rtt threshold, W equals 3; if rtt > rtt threshold, then W is equal to 3 at minimum, and in the case where 500/rtt is greater than 3, W decreases as rtt increases; in addition, the smaller W is, the larger the restricted area is, and the larger W is, the smaller the restricted area is; therefore, to a certain extent, rtt is inversely proportional to W, and rtt is proportional to the limited area, so that the effect of refreshing the I frame with the least frames as possible when the network transmission delay is large can be achieved.
In an embodiment of the present invention, determining the first local image area from the image area of each of the at least two consecutive image frames as the intra-frame coded image area of each of the at least two consecutive image frames includes: and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
This allows an accurate determination of the intra-coded image area of each image frame and finally a complete coverage of the entire image area of a single said image frame.
Specifically, the Height size of the first partial image area may be Height/W line, where Height represents a Height value of an image frame, and W represents the size of the target time window; n +1 referred to above is less than or equal to W. The Height of the second partial image area may also be of the size Height/W line. The width of the first and second local image areas may be the width of the image frame.
Therefore, the non-overlapping of the local image areas can be ensured, and the intra-frame coding area is reduced to the maximum extent so as to reduce the occupied width.
An embodiment of the present invention further provides a decoding method, as shown in fig. 3, including:
step 31: sending an immediate decoding refresh IDR request;
step 32: receiving an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
The encoding end is the encoding device.
The decoding method provided by the embodiment of the invention refreshes IDR request by sending instant decoding; receiving an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame. By the scheme of the embodiment of the application, the effect of complete IDR frame can be achieved by carrying out intra-frame coding on the local image areas of a plurality of image frames after an IDR request is received at a coding end, rather than directly coding at least one image frame into the complete IDR frame for transmission, so that one IDR frame with larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction is well solved.
In the following, the encoding method and the decoding method provided by the embodiments of the present invention are further described, in which the IDR request is sent by a decoding end as an example, the encoding end is a sending end as an example, and the decoding end is a receiving end as an example.
In view of the above technical problems, embodiments of the present invention provide an encoding method and a decoding method, which may be specifically implemented as a fast refresh method for real-time audio and video, where the method requests a sending end to fast refresh an IDR frame (specifically, sends an IDR request to the sending end) when a receiving end counts a large number of packet loss errors. The sending end selects one of 2 modes according to preset conditions: one is to encode the complete IDR frame; the other is to transmit limited P frames or B frames according to network conditions; therefore, the problems of more packet loss, network jitter, image quality reduction and the like caused by bandwidth occupation of huge IDR frames are avoided.
First, the following is a specific example of the content related to the embodiment of the present invention:
(1) the sending end A initiates an audio and video call to the receiving end B.
(2) The receiving end B receives the invitation of the sending end A and establishes a communication link.
(3) The camera of the transmitting end a acquires YUV data (color coded data), encodes the YUV data into an ES Stream (Elementary Stream) according to a video coding standard, and then encapsulates the ES Stream into an RTP (Real-time Transport Protocol) packet to transmit the video data to the receiving end B through an RTP-RTCP (Real-time Transport Control Protocol), and the audio data is processed in the same manner.
(4) And after receiving the audio and video stream of the sending end A, the receiving end B performs decoding and rendering. Meanwhile, the transmitting end of the device to which the receiving end B belongs also collects YUV data, microphone data and the like, performs coding compression and encapsulation, and transmits the stream to the receiving end of the device to which the transmitting end A belongs.
(5) In the case where the network is normal, the above (3) and (4) are always performed. However, it is assumed that in a mobile environment, the receiving-end B network suddenly deteriorates. The receiving end B finds a large amount of lost packets, bit errors, out-of-order frames, and the like, which causes the receiving end B to fail decoding, and to generate phenomena such as screen splash and blocking, and can also be understood as a situation of poor network quality. The receiving end B sends an IDR request to the transmitting end a.
(6) The sending end A processes the IDR request of the receiving end B, can immediately code the current frame into an IDR frame, and sends the IDR frame to the receiving end B; referring to the processing framework shown in fig. 4 (a 21 coding reference a11, a31 coding reference a21 coding reference a32 coding reference a21 coding reference a22 shown in fig. 4, where N is an integer greater than 0), in the case of a weak network (i.e., low network quality), for the case that a receiving end has screen splash, stuck, lost packet, decoding failure, etc., the transmitting end needs to be refreshed quickly to adapt to the case of the weak network. The scheme provides a stable and effective refreshing mechanism, can quickly solve the problems of screen splash and blockage, cannot cause network jitter and more packet loss, and can greatly improve the user experience. The method specifically comprises the following steps:
(1) the sending end A processes the IDR request of the receiving end B.
In this step, the value of the current network rtt packet (i.e. the value of rtt mentioned above) can be obtained by the server. Alternatively, the value of rtt may be determined after the following operation (3), that is, when it is determined that the current image frame meets the limited image frame determination condition.
(2) And starting a network bandwidth estimation module and a limited reference frame module.
(3) The network bandwidth estimation module determines whether the current frame enters the limited reference frame module according to the position (i.e. the arrangement position) of the current frame (the frame to be encoded and transmitted currently, i.e. the current image frame) in the whole GOP and a preset condition. Wherein the preset conditions are as follows: the position proportion corresponding to the arrangement position is M/(T multiplied by fps) is less than or equal to 0.2, or the position proportion is M/(T multiplied by fps) is more than or equal to 0.8; wherein T represents a duration of a GOP (i.e., a duration corresponding to the GOP), for example, T seconds; m represents the number of the current image frame in the GOP (i.e., the number of the current image frame in the GOP); fps represents a frame rate; and if the result obtained by calculation according to the formula is less than or equal to 0.2 or greater than or equal to 0.8, entering a limited reference frame module. Otherwise, the current frame can be immediately encoded into an IDR frame and sent to the receiving end B.
This allows special processing for the previous or the next frame; because the larger IDR frame closer to the previous (or next) frame group occupies larger bandwidth and causes seizure, the special processing can reduce the occupied bandwidth to a certain extent and avoid seizure.
The embodiment of the invention can be as follows: 0.2 XT × fps is not less than 3. The GOP involved is encoder-customized, containing 30 frames or 100 frames, etc.
(4) A restricted reference frame module: the (limited) reference frame sliding window size W (i.e. the size of the target time window) is determined from the value of rtt (i.e. the value of the rtt packet). If rtt ≦ Tm, the sliding window size W ≦ min (3, max (500/rtt, 5)). Where Tm is a predetermined threshold (i.e. the rtt threshold may be greater than or equal to 100ms and less than or equal to 300ms), and represents the time of the delay, and is generally 200 ms. If rtt > Tm, the sliding window size is W ═ max (3,500/rtt). Therefore, the effect of refreshing the I frame by using the least frames as possible when the network transmission delay is large can be realized:
thus, if rtt ≦ Tm, W equals 3; if rtt > Tm, then W is at least equal to 3, and in the case where 500/rtt is greater than 3, W decreases as rtt increases; further, the smaller W, the larger the restricted area, and the larger W, the smaller the restricted area. Therefore, to a certain extent, the value of rtt is inversely proportional to W, and the value of rtt is proportional to the limited area, so that the effect of refreshing the I frame by using the least frames as possible when the network transmission delay is large can be achieved.
(5) A restricted area of the reference frame (P-frame) for each frame within the sliding window (a specific implementation of the restricted area of the image frame described above) is calculated, the image Height of the restricted area being (Height/W) lines. It can also be understood that the size of the reference frame restricted area is determined according to the rtt value in the present scheme.
(6) The encoding continues from the current 0 th frame (i.e. the above-mentioned current image frame) to the W-1 th frame as P or B frames (bi-directional predictive interpolated frames), the following relates to the special encoding of the restricted area:
1) the 0 th frame only has a limited area, specifically an area with the Height from 0 to (Height/W) line and the width of the image frame width; the region coding mode of the current frame is intra-frame coding. And other areas are normally coded (normally coded according to coding requirements). For example, if the 0 th frame is originally a B frame or a P frame, the region outside the limited region is encoded as a B frame or a P frame according to the original principle; wherein, the P frame belongs to the interframe coding of forward prediction, and only refers to the I frame or the P frame which is closest to the P frame; the B frame is a bidirectional predictive coding frame, and inter-frame coding is performed using a preceding I or P frame and a following P frame as reference frames.
2) The 1 st frame has 2 restricted areas, specifically two areas with a Height from 0 to (Height/W) line and from (Height/W) to 2 × (Height/W) line, and the width of the two areas is the area of the image frame width. The first limited area of the current frame adopts interframe coding, and refers to the first limited area of the 0 th frame (interframe coding needs to refer to the previous frame, and the limited area referring to the previous frame is limited). The second restricted area of the current frame is intra-coded. The other regions are normally encoded.
3) The 2 nd frame has 3 restricted areas, specifically three areas having heights from 0 to (Height/W), from (Height/W) to 2 x (Height/W), from 2 x (Height/W) to 3 x (Height/W), and the widths of the three areas are areas of the image frame width. The first and second restricted areas of the current frame are inter-coded with reference to the first and second restricted areas of the 1 st frame. The third restricted area of the current frame is intra-coded. The other regions are normally encoded.
By analogy, the following limited regions from the 3 rd frame to the W-2 th frame and the encoding can be processed similarly as described above;
4) the W-1 th frame has W restricted areas, specifically, areas with heights from 0 to (Height/W), from (Height/W) to 2 (Height/W), from 2 (Height/W) to 3 (Height/W), from (W-1) to (Height/W) to Wx (Height/W), and widths of the W areas are areas with image frame widths. The first, second to W-1 restricted areas of the current frame are inter-coded with reference to the first, second to W-1 restricted areas of the W-2 frame. The W-th restricted area of the current frame is intra-coded. The other regions are normally encoded.
Therefore, block refreshing (intra-frame coding) can be realized, and the coding mode can ensure that the coded data can be normally displayed; the intra-frame coding mode can ensure that data refreshing is realized and subsequent normal display is ensured; and the interframe coding can ensure that the reference is the refreshed data, thereby ensuring the subsequent normal display.
5) Frames beyond the sliding window are coded as normal.
Thus, the scheme provided by the embodiment of the invention can realize that: the quick refreshing can be realized under the condition of not causing network jitter under the weak network; the limited reference frame only carries out intra-frame coding on a specific limited area, and the image quality in the weak network can be guaranteed to be better.
An embodiment of the present invention further provides an encoding apparatus, as shown in fig. 5, including:
a first determining module 51 for determining at least two consecutive image frames from the current group of pictures, GOP, in response to the received instantaneous decoding refresh IDR request;
a second determining module 52, configured to determine the first local image areas from the respective image areas of the at least two consecutive image frames as intra-frame coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame;
a first encoding module 53, configured to, when encoding the at least two consecutive image frames, perform: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
The encoding device provided by the embodiment of the invention responds to the received instantaneous decoding refresh IDR request, and determines at least two continuous image frames from the current group of pictures (GOP); determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame; in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode. By the scheme of the embodiment of the application, after the IDR request is received, intra-frame coding can be performed on the local image areas of the image frames to achieve the effect of a complete IDR frame, instead of directly coding at least one image frame into the complete IDR frame to be transmitted, so that an IDR frame with a larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with a smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, the problems of more packet loss, network jitter, image quality reduction and the like caused by the coding and decoding scheme aiming at the IDR request in the prior art are solved well, and the problem of image quality reduction caused by the coding and decoding scheme aiming at the IDR request is solved well.
Further, the apparatus further comprises: a third determining module, configured to determine second local image regions from respective image regions of a specified image frame, as inter-frame encoded image regions of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area; a second encoding module, configured to, when encoding the specified image frame, perform: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
In the embodiment of the present invention, in response to the received instantaneous decoding refresh IDR request, determining at least two consecutive image frames from a current group of pictures GOP includes: determining an index value characterizing network quality in response to the IDR request; and determining the at least two continuous image frames from the current GOP according to the index value.
Wherein, in response to the IDR request, determining an index value characterizing network quality comprises: responding to the IDR request, and judging whether the current image frame meets the judgment condition of the limited image frame; and if so, determining the index value.
In the embodiment of the present invention, the determining whether the current image frame meets the determination condition of the limited image frame includes: and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
Determining whether the current image frame to be coded meets the judgment condition of the limited image frame according to the arrangement position of the current image frame in the current GOP, wherein the method comprises the following steps: judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back; if yes, determining that the current image frame meets the limited image frame judgment condition; if not, determining that the current image frame does not meet the limited image judgment condition.
In the embodiment of the invention, the index value comprises a round-trip delay rtt value; determining the at least two consecutive image frames from the current GOP according to the index value, including: determining the size of a target time window according to the rtt value and the rtt threshold value; and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
In an embodiment of the present invention, determining the first local image area from the image area of each of the at least two consecutive image frames as the intra-frame coded image area of each of the at least two consecutive image frames includes: and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
The implementation embodiments of the encoding method are all suitable for the embodiment of the encoding device, and the same technical effects can be achieved.
An embodiment of the present invention further provides a decoding apparatus, as shown in fig. 6, including:
a first sending module 61, configured to send an instantaneous decoding refresh IDR request;
a first receiving module 62, configured to receive an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
The decoding device provided by the embodiment of the invention refreshes the IDR request by sending the instant decoding; receiving an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request; the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode; the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame. By the scheme of the embodiment of the application, the effect of complete IDR frame can be achieved by carrying out intra-frame coding on the local image areas of a plurality of image frames after an IDR request is received at a coding end, rather than directly coding at least one image frame into the complete IDR frame for transmission, so that one IDR frame with larger occupied bandwidth is divided into a plurality of local intra-frame coding frames with smaller occupied bandwidth, the occupied bandwidth of the image frames can be reduced, and the problems of more packet loss, network jitter, image quality reduction and the like are avoided; the problem that the encoding and decoding scheme aiming at the IDR request in the prior art is easy to cause image quality reduction is well solved.
The implementation embodiments of the decoding method are all suitable for the embodiment of the decoding device, and the same technical effects can be achieved.
The embodiment of the invention also provides coding equipment, which comprises a memory, a processor and a program which is stored on the memory and can run on the processor; the processor implements the above-described encoding method when executing the program.
The implementation embodiments of the encoding method are all suitable for the embodiment of the encoding device, and the same technical effect can be achieved.
The embodiment of the invention also provides decoding equipment, which comprises a memory, a processor and a program which is stored on the memory and can be operated on the processor; the processor implements the above decoding method when executing the program.
The implementation embodiments of the decoding method are all suitable for the embodiment of the decoding device, and the same technical effect can be achieved.
The embodiment of the invention also provides a readable storage medium, wherein a program is stored on the readable storage medium, and the program realizes the steps in the coding method when being executed by a processor; alternatively, the first and second electrodes may be,
which program, when being executed by a processor, carries out the steps of the decoding method described above.
The implementation embodiments of the encoding method or the decoding method are all applicable to the embodiment of the readable storage medium, and the same technical effects can be achieved.
It should be noted that many of the functional units described in this specification have been referred to as modules, in order to more particularly emphasize their implementation independence.
In embodiments of the present invention, modules may be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be constructed as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different bits which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Likewise, operational data may be identified within the modules and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
When a module can be implemented by software, considering the level of existing hardware technology, a module implemented by software may build a corresponding hardware circuit to implement a corresponding function, without considering cost, and the hardware circuit may include a conventional Very Large Scale Integration (VLSI) circuit or a gate array and an existing semiconductor such as a logic chip, a transistor, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
While the preferred embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.

Claims (21)

1. A method of encoding, comprising:
determining at least two consecutive image frames from a current group of pictures, GOP, in response to a received instantaneous decoding refresh, IDR, request;
determining first local image areas from respective image areas of the at least two consecutive image frames as intra-coded image areas of the respective at least two consecutive image frames; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame;
in encoding the at least two consecutive image frames, performing: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
2. The method of claim 1, further comprising:
respectively determining second local image areas from respective image areas of a specified image frame as respective inter-frame coding image areas of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area;
in encoding the specified image frame, performing: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
3. The method of claim 1, wherein determining at least two consecutive image frames from a current group of pictures (GOP) in response to the received Instantaneous Decode Refresh (IDR) request comprises:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
4. The method of claim 3, wherein determining an indicator value characterizing network quality in response to the IDR request comprises:
responding to the IDR request, and judging whether the current image frame meets the judgment condition of the limited image frame;
and if so, determining the index value.
5. The method of claim 4, wherein determining whether the current image frame meets the limited image frame determination condition comprises:
and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
6. The method according to claim 5, wherein determining whether the current image frame to be encoded meets the restricted image frame determination condition according to the arrangement position of the current image frame in the current GOP includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back;
if yes, determining that the current image frame meets the limited image frame judgment condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
7. The method according to claim 3, wherein the indicator value comprises a value of round trip delay rtt;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
8. The method according to claim 7, wherein determining the first local image area from the respective image areas of the at least two consecutive image frames as the intra-coded image areas of the respective at least two consecutive image frames comprises:
and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
9. A method of decoding, comprising:
sending an immediate decoding refresh IDR request;
receiving an image frame obtained by encoding at least two consecutive image frames by an encoding terminal in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
10. An encoding apparatus, comprising:
a first determining module for determining at least two consecutive image frames from a current group of pictures, GOP, in response to a received instantaneous decoding refresh, IDR, request;
a second determining module, configured to determine, from respective image areas of the at least two consecutive image frames, a first local image area as an intra-coded image area of the at least two consecutive image frames, respectively; wherein the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame;
a first encoding module for performing, when encoding the at least two consecutive image frames: and coding the intra-frame coding image area by adopting an intra-frame coding mode.
11. The apparatus of claim 10, further comprising:
a third determining module, configured to determine second local image regions from respective image regions of a specified image frame, as inter-frame encoded image regions of the specified image frame; the specified image frame is the other image frame except the first image frame in the at least two continuous image frames; the inter-coded image regions of each specified image frame satisfy: the position of the intra-frame coding image area of the image frame which is previous to the appointed image frame is the same in the corresponding image area;
a second encoding module, configured to, when encoding the specified image frame, perform: and coding the inter-frame coding image area by adopting a forward prediction coding mode.
12. The apparatus of claim 10, wherein determining at least two consecutive image frames from a current group of pictures, GOP, in response to the received instantaneous decoding refresh, IDR, request comprises:
determining an index value characterizing network quality in response to the IDR request;
and determining the at least two continuous image frames from the current GOP according to the index value.
13. The apparatus of claim 12, wherein determining an indicator value characterizing network quality in response to the IDR request comprises:
responding to the IDR request, and judging whether the current image frame meets the judgment condition of the limited image frame;
and if so, determining the index value.
14. The apparatus of claim 13, wherein determining whether the current image frame meets the limited image frame determination condition comprises:
and determining whether the current image frame to be coded meets the judgment condition of the limited image frame or not according to the arrangement position of the current image frame in the current GOP.
15. The apparatus according to claim 14, wherein determining whether the current image frame to be encoded meets the limited image frame determination condition according to the arrangement position of the current image frame in the current GOP includes:
judging whether the arrangement position of the current image frame in the current GOP is relatively close to the front or relatively close to the back;
if yes, determining that the current image frame meets the limited image frame judgment condition;
if not, determining that the current image frame does not meet the limited image judgment condition.
16. The apparatus of claim 12, wherein the indicator value comprises a round trip delay rtt value;
determining the at least two consecutive image frames from the current GOP according to the index value, including:
determining the size of a target time window according to the rtt value and the rtt threshold value;
and determining the number of continuous image frames matched with the size of the target time window from the GOP according to the size of the target time window, wherein the continuous image frames are used as the at least two continuous image frames.
17. The apparatus according to claim 16, wherein determining the first local image area from the respective image areas of the at least two consecutive image frames as the intra-coded image areas of the respective at least two consecutive image frames comprises:
and according to the size of the target time window and the height value of the image frame, respectively determining a first local image area from the image areas of the at least two continuous image frames as the intra-frame coding image area of the at least two continuous image frames.
18. A decoding apparatus, comprising:
the first sending module is used for sending an instantaneous decoding and refreshing IDR request;
a first receiving module, configured to receive an image frame obtained by an encoding end encoding at least two consecutive image frames in response to the IDR request;
the encoding end performs, when encoding the at least two consecutive image frames: coding the intra-frame coding image areas of the at least two continuous image frames by adopting an intra-frame coding mode;
the intra-coded image area of each of the at least two consecutive image frames is a first partial image area of the image area of each of the at least two consecutive image frames; the respective intra-coded image areas are located in corresponding image areas that collectively cover the entire image area of a single image frame.
19. An encoding device comprising a memory, a processor, and a program stored on the memory and executable on the processor; characterized in that the processor, when executing the program, implements the coding method according to any one of claims 1 to 8.
20. A decoding device comprising a memory, a processor, and a program stored on the memory and executable on the processor; characterized in that the processor, when executing the program, implements the decoding method of claim 9.
21. A readable storage medium on which a program is stored, characterized in that the program, when executed by a processor, implements the steps in the encoding method according to any one of claims 1 to 8; alternatively, the program realizes the steps in the decoding method according to claim 9 when executed by a processor.
CN202110325525.2A 2021-03-26 2021-03-26 Encoding and decoding method, device, equipment and readable storage medium Active CN112929667B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110325525.2A CN112929667B (en) 2021-03-26 2021-03-26 Encoding and decoding method, device, equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110325525.2A CN112929667B (en) 2021-03-26 2021-03-26 Encoding and decoding method, device, equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN112929667A true CN112929667A (en) 2021-06-08
CN112929667B CN112929667B (en) 2023-04-28

Family

ID=76176183

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110325525.2A Active CN112929667B (en) 2021-03-26 2021-03-26 Encoding and decoding method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN112929667B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024007802A1 (en) * 2022-07-08 2024-01-11 中兴通讯股份有限公司 Video encoding method and apparatus, electronic device and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105163117A (en) * 2015-08-26 2015-12-16 华为技术有限公司 Image coding and processing methods and devices
CN106998328A (en) * 2017-03-30 2017-08-01 北京奇艺世纪科技有限公司 A kind of video transmission method and device
CN110784740A (en) * 2019-11-25 2020-02-11 北京三体云时代科技有限公司 Video processing method, device, server and readable storage medium
US20200396501A1 (en) * 2019-06-11 2020-12-17 Nvidia Corporation Re-encoding predicted picture frames in live video stream applications

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105163117A (en) * 2015-08-26 2015-12-16 华为技术有限公司 Image coding and processing methods and devices
CN106998328A (en) * 2017-03-30 2017-08-01 北京奇艺世纪科技有限公司 A kind of video transmission method and device
US20200396501A1 (en) * 2019-06-11 2020-12-17 Nvidia Corporation Re-encoding predicted picture frames in live video stream applications
CN110784740A (en) * 2019-11-25 2020-02-11 北京三体云时代科技有限公司 Video processing method, device, server and readable storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024007802A1 (en) * 2022-07-08 2024-01-11 中兴通讯股份有限公司 Video encoding method and apparatus, electronic device and storage medium

Also Published As

Publication number Publication date
CN112929667B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
WO2016131223A1 (en) Frame loss method for video frame and video sending apparatus
JP2017208849A (en) Small waiting time rate control system and method
US10142644B2 (en) Decoding frames
CN113068001B (en) Data processing method, device, equipment and medium based on cascade camera
WO2021052500A1 (en) Video image transmission method, sending device, and video call method and device
US10819976B2 (en) Long-term reference for error recovery without back channel
CN113676404A (en) Data transmission method, device, apparatus, storage medium, and program
WO2023142716A1 (en) Encoding method and apparatus, real-time communication method and apparatus, device, and storage medium
CN110996035B (en) Information sending method and device
CN101312536B (en) Video coding method and apparatus for video transmission error control
WO2020237466A1 (en) Video transmission method and apparatus, and aircraft, playback device, and storage medium
CN112929667A (en) Encoding and decoding method, device and equipment and readable storage medium
CN112040234B (en) Video encoding method, video decoding method, video encoding device, video decoding device, electronic equipment and storage medium
CN112437319A (en) Method for switching multiple video streams based on webrtc
US11265583B2 (en) Long-term reference for error recovery in video conferencing system
WO2013071460A1 (en) Reducing amount op data in video encoding
EP4184886B1 (en) Method, system and computer program product for initiating downspeeding in a videoconferencing session
CN114189742B (en) System and method for self-adapting projection frame rate to computer terminal during mobile phone interconnection
US10200694B2 (en) Method and apparatus for response of feedback information during video call
CN116962613A (en) Data transmission method and device, computer equipment and storage medium
CN111602389B (en) Code stream transmission control method, device and computer readable storage medium
KR101942371B1 (en) Apparatus and method for video frame scene change detection and encoding in mobile wireless environment
WO2021140768A1 (en) Transmission device and transmission method
CN117061827B (en) Image frame processing method, device, equipment and storage medium
JPH04115792A (en) Picture signal coding system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant