WO2020043206A1 - Method and apparatus for updating historical candidate list - Google Patents

Method and apparatus for updating historical candidate list Download PDF

Info

Publication number
WO2020043206A1
WO2020043206A1 PCT/CN2019/103841 CN2019103841W WO2020043206A1 WO 2020043206 A1 WO2020043206 A1 WO 2020043206A1 CN 2019103841 W CN2019103841 W CN 2019103841W WO 2020043206 A1 WO2020043206 A1 WO 2020043206A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate list
current block
motion information
historical
historical candidate
Prior art date
Application number
PCT/CN2019/103841
Other languages
French (fr)
Chinese (zh)
Inventor
赵寅
徐巍炜
杨海涛
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020043206A1 publication Critical patent/WO2020043206A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Definitions

  • the present application relates to the technical field of video codecs, and in particular, to a method and device for updating historical candidate list, and corresponding encoders and decoders.
  • Video signals have become the most important way to obtain information in people's daily life due to their intuitiveness and efficiency. Because the video signal contains a large amount of data, it needs to occupy a large amount of transmission bandwidth and storage space. In order to effectively transmit and store video signals, it is necessary to compress and encode the video signals. Video compression technology has become an indispensable key technology in the field of video applications.
  • the embodiment of the present application discloses a historical candidate list updating method, which acquires the motion information of the current block; determines whether the depth information, position of the current block, or the area of all decoded blocks in the slice where the current block is not used to update the historical candidate list is The update condition of the historical candidate list is satisfied; if it is satisfied, the historical candidate list is updated according to the motion information of the current block.
  • the number of updates of the historical candidate list is reduced, and the coding efficiency is improved.
  • an embodiment of the present invention provides a method for updating a historical candidate list, which includes: acquiring motion information of a current block; acquiring depth information of a current block; acquiring a historical candidate list; if the depth information of the current block is less than or If it is equal to a preset threshold, the historical candidate list is updated according to the motion information of the current block.
  • an embodiment of the present invention provides a method for updating a historical candidate list.
  • the method includes: acquiring motion information of a current block; acquiring a position of a current block; acquiring a historical candidate list; Assuming an integer multiple of pixels, the historical candidate list is updated according to the motion information of the current block; wherein the preset position of the current block includes the upper left vertex, the upper boundary midpoint, the center point, or Left border center point.
  • an embodiment of the present invention provides a method for updating a historical candidate list.
  • the method includes: obtaining motion information of a current block; and obtaining a piece of the current block that has not been used to update the historical candidate list since the historical candidate list was last updated. Sum the area of all decoded blocks; obtain a historical candidate list; if the sum of the areas is greater than or equal to a preset threshold, update the historical candidate list according to the motion information of the current block.
  • an embodiment of the present invention provides a historical candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; a second acquisition module for acquiring depth information of the current block; Three acquisition modules are used to acquire the historical candidate list; and an update module is used to update the historical candidate list according to the motion information of the current block if the depth information of the current block is less than or equal to a preset threshold.
  • an embodiment of the present invention provides a history candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; a second acquisition module for acquiring a position of a current block; a third An obtaining module for obtaining a historical candidate list; an updating module for updating the historical candidate list according to the motion information of the current block if the preset position of the current block is located at an integer multiple of a preset pixel;
  • the preset position of the current block includes an upper left vertex, an upper boundary midpoint, a center point, or a left boundary center point of the current block.
  • an embodiment of the present invention provides a history candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; and a second acquisition module for acquiring a top-up video of a current block.
  • the area sum of all decoded blocks that are not used to update the history candidate list after the history candidate list is updated once; a third acquisition module is used to obtain the history candidate list; an update module is used if the area sum is greater than or equal to a preset threshold, Updating the historical candidate list according to the motion information of the current block.
  • an embodiment of the present invention provides a historical candidate list update encoder, which is used to implement the method and apparatus of any of the foregoing aspects.
  • an embodiment of the present invention provides a historical candidate list update decoder, which is used to implement the method and device of any of the foregoing aspects.
  • FIG. 1 is a schematic diagram of a video encoding process
  • FIG. 2 is a schematic diagram of an inter prediction
  • FIG. 3 is a schematic diagram of a video decoding process
  • FIG. 4 is a schematic diagram of a candidate position of motion information
  • FIG. 5 is a method for constructing a candidate list of fusion motion information
  • 6a is a method for updating a history candidate list in the prior art
  • Figure 6b is a video system
  • FIG. 7 is a schematic diagram of a method for updating a history candidate list in the present invention.
  • FIG. 8 is a schematic diagram of a method for updating a history candidate list in the present invention.
  • FIG. 9 is a schematic diagram of a method for updating a history candidate list in the present invention.
  • FIG. 10 is a schematic diagram of an update history list device in the present invention.
  • FIG. 11 is a block diagram showing an example structure of a content supply system 3100 that implements a content distribution service.
  • FIG. 12 is a block diagram showing a structure of an example of a terminal device.
  • the process mainly includes intra prediction (Intra Prediction), inter prediction (Inter Prediction), transform (Transform), quantization (Quantization), entropy encoding (Entropy encoding), in-loop filtering (in-loop filtering) (mainly de-blocking filtering).
  • intra prediction or inter prediction is performed, and after the residuals are obtained, transform quantization is performed, and finally entropy coding is performed and a code stream is output.
  • the coding block is an M * N-sized array composed of pixels (M may be equal to N or not equal to N), and the pixel value of each pixel point position is known.
  • Intra prediction refers to using the pixel values of pixels in the reconstructed area in the current image to predict the pixel values of pixels in the current coding block.
  • Inter prediction is to find a matching reference block for the current coding block in the current image in the reconstructed image, and use the pixel value of the pixel point in the reference block as the prediction information or prediction of the pixel value of the pixel point in the current coding block. Value (hereafter information and value are not distinguished), this process is called motion estimation (ME) (as shown in Figure 2), and the motion information of the current coded block is transmitted.
  • ME motion estimation
  • the motion information of the current coding block includes indication information of the prediction direction (usually forward prediction, backward prediction, or bidirectional prediction), one or two motion vectors (Motion vector, MV) pointing to the reference block, And indication information (usually referred to as a reference frame index) of a picture in which a reference block is located.
  • Forward prediction means that the current coding block selects a reference image from the forward reference image set to obtain a reference block.
  • Backward prediction means that the current coding block selects a reference image from the backward reference image set to obtain a reference block.
  • Bidirectional prediction refers to selecting a reference image from the forward and backward reference image sets to obtain a reference block.
  • the bidirectional prediction method there are two reference blocks in the current coding block, each of which requires a motion vector and a reference frame index to indicate, and then the pixels in the current block are determined according to the pixel values of the pixels in the two reference blocks. The predicted value of the pixel value.
  • the motion estimation process needs to try multiple reference blocks in the reference image for the current coding block. Which one or several reference blocks are ultimately used for prediction is determined using rate-distortion optimization (RDO) or other methods.
  • RDO rate-distortion optimization
  • the pixel information of the pixels in the current coding block is subtracted from the corresponding prediction information to obtain residual information. Then, discrete cosine transform (DCT) and other methods The difference information is transformed, and then the code stream is obtained by using quantized entropy coding. After the prediction signal is added with the reconstructed residual signal, further filtering operations are required to obtain a reconstructed signal and use it as a reference signal for subsequent encoding.
  • DCT discrete cosine transform
  • Decoding is equivalent to the reverse process of encoding. For example, as shown in FIG. 3, first, entropy decoding and inverse quantization and inverse transformation are used to obtain residual information.
  • the decoded code stream determines whether the current coding block uses intra prediction or inter prediction. If it is intra prediction, the pixel information of pixels in the surrounding reconstructed area is used to construct prediction information according to the used intra prediction method. If it is inter prediction, you need to parse out the motion information, use the parsed motion information to determine the reference block in the reconstructed image, and use the pixel values of the pixels in the block as prediction information. This process is called motion compensation (Motion Compensation, MC). Reconstruction information can be obtained by using prediction information plus residual information through a filtering operation.
  • Motion Compensation Motion Compensation
  • Inter prediction is a prediction technique based on motion compensation (see Figure 2).
  • inter-prediction coding because the same objects in adjacent frames of the image have a certain time-domain correlation, each frame of the image sequence can be divided into many non-overlapping blocks, and all pixels in the block are considered to have the same motion.
  • the main processing is to determine the motion information of the current block, obtain a reference image block from a reference frame according to the motion information, and generate a predicted image of the current block.
  • Motion information includes inter prediction direction, reference frame index (ref_idx), motion vector (MV, etc.).
  • Inter prediction indicates the prediction direction of the current block using forward prediction, backward prediction, or bidirectional prediction through the inter prediction direction
  • the reference frame is indicated by the reference index
  • the current vector is indicated by the motion vector.
  • the reference block of the current block in the reference frame is offset from the position of the current block in the current frame.
  • the motion vector indicates a displacement vector of a reference image block used to predict the current block relative to the current block in the reference frame, so one motion vector corresponds to one reference image block.
  • H.265 / HEVC, H.266 / VVC and other video coding standards divide a frame of image into non-overlapping Coding Tree Units (CTUs).
  • a CTU is divided into one or more codes.
  • One CU contains coding information, including information such as prediction mode and transform coefficients.
  • Decoding end Perform corresponding prediction, inverse quantization, inverse transform and other decoding processing on the CU according to the encoded information to generate a reconstructed image corresponding to the CU.
  • motion information occupies a large amount of data.
  • motion information is usually transmitted in a predictive manner.
  • intermvp such as but not limited to AMVP in H.265
  • merge such as but not limited to merge
  • the transmitted motion information includes: inter prediction direction (forward, backward, or bidirectional), reference frame index, motion vector prediction value index, and motion vector residual value.
  • inter prediction direction forward, backward, or bidirectional
  • reference frame index reference frame index
  • motion vector prediction value index motion vector residual value
  • motion vector residual value motion vector (difference, MVD)
  • the motion vector prediction may include multiple prediction values.
  • the motion vector prediction candidate list mvp candidate list
  • MVP index motion vector prediction index
  • a merge motion candidate list is constructed on the encoding side and the decoding side in the same way, the index is passed to the decoding side, and the merge index is transmitted in the code stream.
  • the motion information in the candidate list of motion information (candidate list) is usually obtained from its spatial domain neighboring blocks or time domain blocks in the reference frame.
  • the candidate motion information obtained from the motion information of the image blocks adjacent to the current block is called space.
  • candidate motion information obtained from the motion information of the image blocks adjacent to the current block is called space.
  • candidate motion information of the corresponding position image block obtained from the current image in the reference image is called temporal candidate.
  • the current block spatial and time domain candidates of HEVC and VTM are shown in Figure 4, where the spatial candidates include motion information of A0, A1, B0, B1, and B2, and the time domain candidates include T0 and T1 (T0 and T1 are both in the reference frame).
  • the JVET-K0104 proposal proposes a method of adding history candidates to the fusion motion information candidate list, which increases the number of fusion motion information candidates in merge / skip mode and improves prediction efficiency.
  • the proposal also proposes a method of adding history candidates to the motion vector prediction candidate list (including motion vectors), which increases the number of motion vector prediction candidates in the InterMVP mode and improves prediction efficiency.
  • the history candidate list is composed of history candidates, where the history candidates are motion information of a previously encoded / decoded block (a coded block that has been encoded or decoded before the current block in the current slice). It can be understood as a list that exists all the time. After encoding or decoding an image block, the motion information of the image block is added.
  • the historical candidate list can contain at most N motion information, where N is a preset value.
  • the JVET-K0104 proposal introduced the use of history candidate list and the construction method of history candidate list.
  • the method of constructing the candidate list of fused motion information added to the historical candidate can also be understood as the method of using the historical candidate list, as follows:
  • Step 11 The spatial candidate and the time domain candidate of the current block are created according to predetermined rules (for example, according to the addition order and check strategy of HEVC or VTM), and a list (mi) of the fusion motion information candidate of the current block is created.
  • the spatial candidate may include A0, A1, B0, B1, and B2, and the time domain candidate may include T0 and T1.
  • the spatial candidate and the temporal candidate may be finally put into the fusion motion information candidate list, which may be a preset number of candidates, such as all or part.
  • the airspace candidate is placed first in the fusion motion information candidate list, and then the time domain candidate is placed.
  • time-domain candidates may also include candidates provided by the adaptive time-domain motion vector prediction (ATMVP) technology.
  • ATMVP adaptive time-domain motion vector prediction
  • Step 12 Add historical candidates in the historical candidate list list (h) to the fusion motion information candidate list in a predetermined order.
  • JVET-K0104 checks a preset number of historical candidates in the order from the tail to the head of the history candidate list. As shown in FIG. 5, starting from the historical candidate at the tail of the historical candidate list, check whether it is the same as the fused motion information candidate in the fused motion information candidate list obtained in step 11. Preset position. The preset position may include all spatial domain candidates or be placed after time domain candidates. If they are the same, the next historical candidate in the historical candidate list is checked, and so on, until the number of MVs in the fused motion information candidate list reaches a preset number, and no historical candidate check and addition is performed.
  • Figure 5 Schematic diagram of historical candidates joining the candidate list of fusion motion information
  • Step 13 As an option, if all historical candidates in the historical candidate list have been traversed, and the number of MVs in the fusion motion information candidate list has not reached a preset number, other types of fusion motion information candidates may be added. , Such as bi-predictive candidates and zero motion vector candidates.
  • the construction method of the motion vector prediction candidate list added to the historical candidate can also be understood as the use method of the historical candidate list, which can be as follows:
  • Step 21 Add the spatial candidate and the time domain candidate of the current block to the motion vector prediction candidate list (mvp) of the current block, the method of which is the same as the method in HEVC.
  • the spatial candidates include A0, A1, B0, B1, and B2, and the time domain candidates include T0 and T1.
  • Step 22 Add historical candidates in the historical candidate list list (h) (or Hlist) to the motion vector prediction candidate list, and check a preset number of historical candidates in the order from the tail to the head of the history candidate list .
  • h historical candidate list list
  • Hlist Hlist
  • Candidate list reference frame index Motion vectors using the same reference frame index will be added to the motion vector prediction candidate list. If they are not the same, the next historical candidate in the historical candidate list will be checked, and so on, until the motion vector prediction candidate list The number of MVs in the game reaches a preset number, and the history candidate check and addition are no longer performed.
  • Step 23 Add other types of motion vector candidates, such as zero motion vector candidates.
  • the historical candidate list is constructed using the motion information of the coded / decoded blocks in the current frame, and the historical candidate list is accessed in a first-in-first-out manner.
  • the overall historical candidate list in the encoding / decoding side is constructed as follows:
  • Step 31 Initialize the history candidate list at the beginning of slice decoding and clear it. It can be understood that, for each new slice, the history candidate list is initialized and cleared when decoding is started.
  • Step 32 Decode the current CU. If the current CU or current block is a merge / skip inter prediction mode, it may correspond to a candidate list of fused motion information. If the current CU or current block is an inter inter prediction mode, it may correspond to motion vector prediction. Candidate list; meanwhile, historical candidates in the historical candidate list are added to the fusion motion information candidate list or the motion vector prediction candidate list.
  • the motion information of the current block is determined according to the fusion index and the fusion motion information candidate list carried in the code stream.
  • the motion information of the current block is determined according to the inter prediction direction, reference frame index, motion vector prediction value index, motion vector prediction candidate list, and motion vector residual value transmitted in the bitstream.
  • Step 33 After decoding the current CU or current block, the motion information of the current block is added as a new historical candidate to the historical candidate list, and then the historical candidate list is updated, as shown in FIG. 6a.
  • the motion information of the current block is compared with historical candidates in the historical candidate list. If a historical candidate (for example, MV2 in FIG. 6a) is the same as the motion information of the current block, remove the historical candidate MV2 from the history list and add the motion information of the current block to the tail of the historical candidate list; Although MV2 has been removed, the ranking position of the same motion information in the historical candidate list has changed.
  • a historical candidate for example, MV2 in FIG. 6a
  • a historical candidate (such as MV2 in Figure 6a) is different from the motion information of the current block, check the number of historical candidates in the historical candidate list. If the number of historical candidates in the list exceeds a preset value, change the The historical candidate at the head of the historical candidate list is removed, and the motion information of the current block is added to the tail of the historical candidate list.
  • steps 32-33 For each block in a slice, perform steps 32-33 until all the blocks in the slice have been traversed; for the next slice, go to steps 31-33.
  • a slice encoding or decoding is used to start the construction and use of the historical candidate list. Assume that the same motion information is not available during the update process:
  • step 31 uses: Hlist (NULL); ...; step 34 builds: Hlist (V1)
  • step 31 uses: Hlist (V1); ...; step 34 builds: Hlist (V1, V2)
  • step 31 uses: Hlist (V1, V2); ...; step 34 builds: Hlist (V1, V2, V3) ...;
  • step 31 uses: Hlist (V1, V2, ..., V6); ...; step 34 builds: Hlist (V1, V2, ..., V7);
  • step 31 uses: Hlist (V1, V2, ..., V7); ...; Step 34 builds: Hlist (V1, V2, ..., V8);
  • step 31 uses: Hlist (V1, V2, ..., V8); ...; Step 34 builds: Hlist (V2, V3, ..., V9);
  • the historical candidate list grid is updated. Especially, when the historical candidate list is long, the time consumption of constructing and updating it will be longer.
  • the method uses a history candidate list (assuming length K, that is, contains a maximum of K historical candidates) to construct a fusion motion information candidate list (assuming length L, that is, contains a maximum of L fusion motion information candidates) or a motion vector prediction candidate list (Assuming that the length is J, that is, including at most L motion vector prediction candidates), the subsequent operations respectively require K * J * L duplicate item searches, resulting in a substantial increase in program execution time.
  • the present invention proposes an improved method for updating and using a history candidate list, and the history candidate list is updated only under specific conditions. It also enables some historical candidates to be skipped when constructing a fused motion information candidate list or a motion vector prediction candidate list using a historical candidate list. Reduce complexity.
  • the system framework of the present invention is shown in Figure 6b.
  • the present invention is mainly located in the video encoding and video decoding in the system framework.
  • Existing video transmission systems usually consist of these parts: capture, encode, send, receive, decode, and display.
  • the acquisition module contains a camera or camera group and pre-processing to convert the light signal into a digitized video sequence.
  • the video sequence is then encoded by an encoder and converted into a code stream.
  • the code stream is then sent by the sending module to the receiving module via the network, and converted into a code stream by the receiving module, and then decoded and reconstructed by the decoder into a video sequence.
  • the reconstructed video sequence is sent to a display device for display after post-processing such as rendering.
  • Existing video transmission systems usually consist of these parts: capture, encode, send, receive, decode, and display.
  • the acquisition module contains a camera or camera group and pre-processing to convert the light signal into a digitized video sequence.
  • the video sequence is then encoded by an encoder and converted into a code stream.
  • the code stream is then sent by the sending module to the receiving module via the network, and converted into a code stream by the receiving module, and then decoded and reconstructed by the decoder into a video sequence.
  • the reconstructed video sequence is sent to a display device for display after post-processing such as rendering.
  • the present invention is mainly located in the video encoding and video decoding in the system framework.
  • FIGS. 1 and 3 An application scenario of the present invention is shown in a video coding and decoding system based on a hybrid coding framework shown in FIGS. 1 and 3.
  • it mainly includes intra prediction (Intra Prediction), inter prediction (Inter Prediction), transform (Transform), quantization (Quantization), entropy encoding (Entropy encoding), and in-loop filtering ( Loop filter) and other steps, respectively complete the prediction block from the neighboring pixels of the current frame, calculate the MV information and obtain the prediction block from the reference frame, transform the residual from the pixel domain to the transform domain, compress the transform domain coefficients, and perform the encoding information Compression, post-processing of the reconstructed image, etc.
  • the decoding system As shown in Figure 3, it is equivalent to the inverse process of encoding.
  • the present invention is mainly applied to inter prediction in a video encoding or decoding system. It is applied to motion vector prediction in an inter prediction mode, and the processing of updating and using the historical candidate list is the same at the encoding end and the decoding end.
  • the following embodiment uses the decoder as an example to describe the description.
  • those skilled in the art should be able to reproduce the relevant encoding method through the method and device description of the decoder. The encoder will not be described in detail in this application.
  • the present invention relates to a method for updating a history candidate list, and a process of decoding an image block using the history candidate in the updated history candidate list.
  • the length K of the historical candidate list is a preset value, that is, the number of historical candidate motion information candidates included in the historical candidate list after construction is K, and K is a positive integer greater than 0.
  • the history candidate list initialization process can be the existing technology. This process can be performed in the same way as the JVET-K0104 proposal, that is, when the SLICE starts decoding, the history candidate list is emptied, or other history candidate list initialization can be used. The method is not limited by the present invention.
  • the above-mentioned decoding process includes steps 41 to 43, and an image block being decoded is referred to as a current block.
  • Step 41 Parse the inter prediction mode of the current block. If the current block is in merge / skip mode, generate a candidate list of fused motion information. If the current CU or current block is in the Inter MVP mode, a motion vector prediction candidate list is generated. Historical candidates in the historical candidate list are added to the fused motion information candidate list or the motion vector prediction candidate list according to a preset rule.
  • the method for generating a candidate list of fused motion information may adopt the above steps 11 to 13.
  • the process may be performed using a method in HEVC or VTM, or other methods for generating a candidate list of fused motion information.
  • a preset number of historical candidates can be checked and added in the order from the tail to the head of the history candidate list.
  • the method for generating a motion vector prediction candidate list may adopt steps 21 to 23.
  • the method in HEVC or VTM may be adopted, and other methods for generating a motion vector prediction candidate list may also be adopted, which is not limited in the present invention.
  • the method in the JVET-K0104 proposal can be followed. Check and add a preset number of historical candidates in order from the tail to the head of the historical candidate list. Only historical candidates that use the same reference frame index as the MVP mode target reference frame index will be added to the motion vector prediction candidate list. After the historical candidate is added to the motion vector prediction candidate list in the motion vector prediction candidate list, it is not limited in the present invention.
  • some decoding blocks can use steps 11 to 13 and other decoding blocks can use the above steps 21 to 23, and there may be some decoding blocks that do not build motion.
  • the vector candidate list does not build a candidate list of fused motion information, and can be configured according to preset rules and system performance.
  • Step 42 Acquire the motion information of the current block.
  • the motion information of the current block is determined according to the fusion index and the fusion motion information candidate list carried in the code stream.
  • the current block motion information is determined according to the inter prediction direction, reference frame index, motion vector prediction value index, motion vector prediction candidate list, and motion vector residual value transmitted in the bitstream.
  • Step 43 Determine whether to update the historical candidate list according to the motion information of the current block by determining some conditions.
  • the achievable manner includes at least three parallel implementation manners of step 43a, step 43b, and step 43c (please refer to Figs. 7, 8, and 9 respectively).
  • Step 43a Steps 43a1, 43a2 and 43a3 may be included.
  • the historical candidate list is not updated with the motion information of the current block.
  • the preset value H is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder.
  • Depth value is a positive integer greater than 0 and is less than the maximum allowed by the encoder or
  • the historical candidate list is selectively updated at the CU level according to the depth value of the current block.
  • Step 43b It may include three steps of steps 43b1, 43b2, and 43b3.
  • the preset position can be the top left corner of the current block (relative coordinate offset is (0,0)), or it can be the center point of the current block (relative coordinate offset is (the current block width divided by two, the current block height divided by To two)).
  • the method in the JVET-K0104 proposal can be used, or other methods can be used to historical candidates based on the motion information of the current block.
  • the list is updated.
  • the motion information of the current block is compared with the historical candidate in the historical candidate list; if a historical candidate is the same as the current block motion information, the historical candidate is removed from the history. Removed from the candidate list. Then, check the history candidate list size. If the list size exceeds a preset size, remove the historical candidate at the head of the list. Finally, the motion information of the current block is added to the historical candidate list.
  • the historical candidate list is selectively updated at the CU level according to the coordinates of a point at a preset position in the current block.
  • Step 43c It may include three steps of steps 43c1, 43c2, and 43c3.
  • the cumulative unupdated coding block area S is smaller than a preset value, the area of the current block is added to the cumulative unupdated coding block area, and the preset value is a positive integer greater than 0. If the cumulative unupdated coding block area S is greater than or equal to the preset value, the cumulative unupdated coding block area is cleared to 0, and the method in the JVET-K0104 proposal may be used, or other methods may be used to use the motion information of the current block to history The candidate list is updated.
  • the motion information of the current block is compared with the historical candidate in the historical candidate list; if a historical candidate is the same as the current block motion information, the historical candidate is removed from the history. Removed from the candidate list. Then, check the history candidate list size. If the list size exceeds a preset size, remove the historical candidate at the head of the list. Finally, the motion information of the current block is added to the historical candidate list.
  • the method may further include step 44.
  • Step 44 Obtain an inter prediction image of the current block according to the motion information, and add the inter prediction image and the residual image to obtain a reconstructed image of the current block.
  • step 43 if the motion information of the current block does not update the historical candidate list, the current historical candidate list is still used as the historical candidate list of the next decoding block, and steps 41 and 42 are performed. If the historical candidate list is updated with the motion information of the current block, the new historical candidate list will be used as the historical candidate list of the next decoded block, and steps 41 and 42 are performed. The rest can be deduced by analogy.
  • the method for determining whether the current block motion information is the same as a certain historical candidate in the historical candidate list is not limited. It can be that the two pieces of motion information are completely the same, or that the two pieces of motion information are the same after some processing, for example, the results of two motion vectors shifted to the right by 2 bits are the same.
  • the method for updating the history list provided by the present invention, when the historical candidate list is added to the fusion motion information candidate list or the motion vector prediction candidate list, part of the historical candidates are selectively skipped, and the historical candidate list is reduced to be added to the fusion motion information candidate list or The number of duplicate check operations when motion vector prediction is in the candidate list. Under the premise of considerable coding and compression efficiency, the codec time is reduced. It can reduce the number of updates of the historical candidate list and increase the coding efficiency without adding additional storage areas and having a comparable coding efficiency.
  • an embodiment of the present application provides a device for updating a historical candidate list.
  • the device includes:
  • a first acquisition module 501 configured to acquire motion information of a current block
  • a third acquisition module 503, configured to acquire a history candidate list
  • An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the depth information of the current block is less than or equal to a preset threshold.
  • an embodiment of the present application provides a device for updating a historical candidate list.
  • the device includes:
  • a first acquisition module 501 configured to acquire motion information of a current block
  • a third acquisition module 503, configured to acquire a history candidate list
  • An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the preset position of the current block is an integer multiple of a preset pixel; wherein the preset of the current block is The position includes the upper left vertex, the upper boundary midpoint, the center point, or the left boundary center point of the current block.
  • an embodiment of the present application provides a device for updating a historical candidate list.
  • the device includes:
  • a first acquisition module 501 configured to acquire motion information of a current block
  • a second obtaining module 502 configured to obtain the total area of all decoded blocks in the slice where the current block is not used to update the historical candidate list since the historical candidate list was last updated;
  • a third acquisition module 503, configured to acquire a history candidate list
  • An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the total area is greater than or equal to a preset threshold.
  • the first acquisition module 501 is configured to execute the methods corresponding to steps 41 and 42 and a method that can be partially equivalently replaced;
  • the second acquisition module 502 is configured to execute the methods corresponding to step 43a1 or 43b1 or 43c1 and can be applied to A method for performing partial equivalent replacement thereof;
  • a third obtaining module for performing a method corresponding to step 43a2 or 43b2 or 43c2 and a method capable of performing partial equivalent replacement thereof;
  • an updating module 505 for performing a method corresponding to step 43a3 or 43b3 or 43c3 And methods that can be partially replaced equivalently.
  • An embodiment of the present application provides a coding device, which includes a non-volatile memory and a processor coupled to each other, and the processor calls program code stored in the memory to perform step 41, step 42, and step 43a, or The method of step 43b or step 43c and the equivalent method.
  • An embodiment of the present application provides a decoding device, including: a non-volatile memory and a processor coupled to each other, the processor calling program code stored in the memory to perform step 41, step 42, and step 43a, or The method of step 43b or step 43c and the equivalent method.
  • An embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores program code, where the program code includes steps for performing step 41, step 42, and step 43a, or step 43b or step 43c. Instructions and some or all of the steps in the equivalent method.
  • An embodiment of the present application provides a computer program product, and when the computer program product runs on a computer, the computer causes the computer to execute the method in step 41, step 42, and step 43a, or step 43b or step 43c and equivalent Some or all steps.
  • FIG. 11 is a block diagram showing a content supply system 3100 for implementing a content distribution service.
  • This content provisioning system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126.
  • the capture device 3102 communicates with the terminal device 3106 through a communication link 3104.
  • the communication link may include the communication channel 13 described above.
  • the communication link 3104 includes, but is not limited to, WIFI, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.
  • the capture device 3102 generates data, and can encode the data by an encoding method as shown in the above embodiments.
  • the capture device 3102 may distribute the data to a streaming media server (not shown in FIG. 11), the server encodes the data and transmits the encoded data to the terminal device 3106.
  • the capturing device 3102 includes, but is not limited to, a camera, a smartphone or a tablet computer, a computer or a notebook computer, a video conference system, a PDA, a vehicle-mounted device, or a combination of any of them.
  • the capture device 3102 may include the source device 12 as described above.
  • the video encoder 20 included in the capture device 3102 may actually perform a video encoding process.
  • the audio encoder included in the capture device 3102 can actually perform audio encoding processing.
  • the capture device 3102 distributes the encoded video data and the encoded audio data by multiplexing the encoded video data and the encoded audio data together.
  • encoded audio data and encoded video data are not multiplexed.
  • the capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106, respectively.
  • the terminal device 310 receives and reproduces the encoded data.
  • the terminal device 3106 can be a device with data receiving and recovery capabilities, such as a smart phone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112 , TV 3114, set-top box (STB) 3116, video conference system 3118, video surveillance system 3120, personal digital assistant (PDA) 3122, in-vehicle equipment 3124, or above devices capable of decoding the above-mentioned encoded data Any combination of them etc.
  • the terminal device 3106 may include the destination device 14 as described above.
  • the encoded data includes a video
  • the video decoder 30 included in the terminal device is prioritized to perform video decoding.
  • the audio decoder included in the terminal device is prioritized to perform audio decoding processing.
  • the terminal device can feed the decoded data to its display.
  • NVR network video recorders
  • DVR digital video recorders
  • TV 3114 TV 3114
  • PDA personal digital assistant
  • the terminal device can feed the decoded data to its display.
  • an external display 3126 is connected to receive and display decoded data.
  • an image encoding device or an image decoding device as shown in the above embodiment can be used.
  • FIG. 12 is a diagram showing a configuration of an example of the terminal device 3106.
  • the protocol processing unit 3202 analyzes the transmission protocol of the stream.
  • the protocols include, but are not limited to, Real-Time Streaming Protocol (RTSP), HyperText Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH , Real-time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.
  • a stream file is generated.
  • the file is output to the demultiplexing unit 3204.
  • the demultiplexing unit 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for other practical scenarios, such as in a video conference system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is transmitted to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.
  • the video decoder 3206 includes the video decoder 30 as described in the above embodiment, and decodes the video ES to generate a video frame by using the decoding method as shown in the above embodiment, and feeds this data to the synchronization unit 3212.
  • the audio decoder 3208 decodes the audio ES to generate an audio frame, and feeds this data to the synchronization unit 3212.
  • the video frames may be stored in a buffer (not shown in FIG. 12) before the video frames are fed to the synchronization unit 3212.
  • the audio frames may be stored in a buffer (not shown in FIG. 12) before the audio frames are fed to the synchronization unit 3212.
  • the synchronization unit 3212 synchronizes the video frame and the audio frame, and provides the video / audio to the video / audio display 3214.
  • the synchronization unit 3212 synchronizes the presentation of video and audio information.
  • the information may be encoded in syntax using timestamps related to the presentation of the encoded audio and visual data and timestamps related to the transmission of the data stream itself.
  • the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frame and the audio frame, and provides the video / audio / subtitle to the video / audio / subtitle display 3216.
  • the present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiment can be incorporated into other systems, such as a car system.
  • Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol) .
  • computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave.
  • a data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used to implement the techniques described in this application.
  • the computer program product may include a computer-readable medium.
  • such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or may be used to store instructions or data structures Any form of desired program code and any other medium accessible by a computer.
  • any connection is properly termed a computer-readable medium.
  • a coaxial cable is used to transmit instructions from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media.
  • the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are instead directed to non-transitory tangible storage media.
  • magnetic disks and compact discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, where disks typically reproduce data magnetically, and optical discs use lasers to reproduce optically data. Combinations of the above should also be included within the scope of computer-readable media.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • processor may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein.
  • functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or Into the combined codec.
  • the techniques can be fully implemented in one or more circuits or logic elements.
  • the techniques of this application may be implemented in a wide variety of devices or devices, including a wireless handset, an integrated circuit (IC), or a group of ICs (eg, a chipset).
  • IC integrated circuit
  • Various components, modules, or units are described in this application to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units.
  • the various units may be combined in a codec hardware unit in combination with suitable software and / or firmware, or through interoperable hardware units (including one or more processors as described above) provide.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Disclosed is a method for updating a historical candidate list. The method comprises: acquiring motion information of a current block; determining whether depth information and the position of the current block or the total area of all decode blocks, not used for updating a historical candidate list, in a slice where the current block is located meet an update condition for the historical candidate list; and if so, updating the historical candidate list according to the motion information of the current block. Compared with the prior art, the present application reduces the number of updates of a historical candidate list and improves the coding efficiency.

Description

一种历史候选列表更新方法与装置Method and device for updating historical candidate list
本申请要求于2018年8月30日提交中国国家知识产权局、申请号为201810999655.2、申请名称为“一种历史候选列表更新方法与装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed on August 30, 2018 with the State Intellectual Property Office of China, with the application number 201810999655.2, and the application name being "A Method and Device for Updating Historical Candidate List", the entire contents of which are incorporated by reference In this application.
技术领域Technical field
本申请涉及视频编解码技术领域，尤其涉及一种历史候选列表更新方法、装置以及相应的编码器和解码器The present application relates to the technical field of video codecs, and in particular, to a method and device for updating historical candidate list, and corresponding encoders and decoders.
背景技术Background technique
随着信息技术的发展，高清晰度电视，网络会议，IPTV，3D电视等视频业务迅速发展，视频信号以其直观性和高效性等优势成为人们日常生活中获取信息最主要的方式。由于视频信号包含的数据量大，需要占用大量的传输带宽和存储空间。为了有效的传输和存储视频信号，需要对视频信号进行压缩编码，视频压缩技术越来越成为视频应用领域不可或缺的关键技术。With the development of information technology, high-definition television, web conferencing, IPTV, 3D TV and other video services have developed rapidly. Video signals have become the most important way to obtain information in people's daily life due to their intuitiveness and efficiency. Because the video signal contains a large amount of data, it needs to occupy a large amount of transmission bandwidth and storage space. In order to effectively transmit and store video signals, it is necessary to compress and encode the video signals. Video compression technology has become an indispensable key technology in the field of video applications.
发明内容Summary of the Invention
本申请实施例公开了一种历史候选列表更新方法，获取当前块的运动信息；判断当前块的深度信息、位置或当前块所在片中未用于更新历史候选列表的所有解码块的面积总和是否满足历史候选列表的更新条件；如果满足，则根据当前块的运动信息对历史候选列表进行更新。相比于现有技术，减少了历史候选列表的更新次数，提高了编码效率。The embodiment of the present application discloses a historical candidate list updating method, which acquires the motion information of the current block; determines whether the depth information, position of the current block, or the area of all decoded blocks in the slice where the current block is not used to update the historical candidate list is The update condition of the historical candidate list is satisfied; if it is satisfied, the historical candidate list is updated according to the motion information of the current block. Compared with the prior art, the number of updates of the historical candidate list is reduced, and the coding efficiency is improved.
第一方面，本发明实施例提供一种历史候选列表更新方法，该方法包括：获取当前块的运动信息；获取当前块的深度信息；获取历史候选列表；若所述当前块的深度信息小于或等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。In a first aspect, an embodiment of the present invention provides a method for updating a historical candidate list, which includes: acquiring motion information of a current block; acquiring depth information of a current block; acquiring a historical candidate list; if the depth information of the current block is less than or If it is equal to a preset threshold, the historical candidate list is updated according to the motion information of the current block.
第二方面，本发明实施例提供一种历史候选列表更新方法，该方法包括：获取当前块的运动信息；获取当前块的位置；获取历史候选列表；若所述当前块的预设位置位于预设像素的整数倍，则根据所述当前块的运动信息对所述历史候选列表进行更新；其中，所述当前块的预设位置包括当前块的左上顶点、上边界中点、中心点、或左边界中心点。In a second aspect, an embodiment of the present invention provides a method for updating a historical candidate list. The method includes: acquiring motion information of a current block; acquiring a position of a current block; acquiring a historical candidate list; Assuming an integer multiple of pixels, the historical candidate list is updated according to the motion information of the current block; wherein the preset position of the current block includes the upper left vertex, the upper boundary midpoint, the center point, or Left border center point.
第三方面，本发明实施例提供一种历史候选列表更新方法，该方法包括：获取当前块的运动信息；获取当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和；获取历史候选列表；若所述面积总和大于等于预 设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。According to a third aspect, an embodiment of the present invention provides a method for updating a historical candidate list. The method includes: obtaining motion information of a current block; and obtaining a piece of the current block that has not been used to update the historical candidate list since the historical candidate list was last updated. Sum the area of all decoded blocks; obtain a historical candidate list; if the sum of the areas is greater than or equal to a preset threshold, update the historical candidate list according to the motion information of the current block.
第四方面，本发明实施例提供一种历史候选列表更新装置，该装置包括：第一获取模块，用于获取当前块的运动信息；第二获取模块，用于获取当前块的深度信息；第三获取模块，用于获取历史候选列表；更新模块，用于若所述当前块的深度信息小于或等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。According to a fourth aspect, an embodiment of the present invention provides a historical candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; a second acquisition module for acquiring depth information of the current block; Three acquisition modules are used to acquire the historical candidate list; and an update module is used to update the historical candidate list according to the motion information of the current block if the depth information of the current block is less than or equal to a preset threshold.
第五方面，本发明实施例提供一种历史候选列表更新装置，该装置包括：第一获取模块，用于获取当前块的运动信息；第二获取模块，用于获取当前块的位置；第三获取模块，用于获取历史候选列表；更新模块，用于若所述当前块的预设位置位于预设像素的整数倍，则根据所述当前块的运动信息对所述历史候选列表进行更新；其中，所述当前块的预设位置包括当前块的左上顶点、上边界中点、中心点、或左边界中心点。In a fifth aspect, an embodiment of the present invention provides a history candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; a second acquisition module for acquiring a position of a current block; a third An obtaining module for obtaining a historical candidate list; an updating module for updating the historical candidate list according to the motion information of the current block if the preset position of the current block is located at an integer multiple of a preset pixel; The preset position of the current block includes an upper left vertex, an upper boundary midpoint, a center point, or a left boundary center point of the current block.
第六方面，本发明实施例提供一种历史候选列表更新装置，该装置包括：第一获取模块，用于获取当前块的运动信息；第二获取模块，用于获取当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和；第三获取模块，用于获取历史候选列表；更新模块，用于若所述面积总和大于等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。According to a sixth aspect, an embodiment of the present invention provides a history candidate list updating device, which includes: a first acquisition module for acquiring motion information of a current block; and a second acquisition module for acquiring a top-up video of a current block. The area sum of all decoded blocks that are not used to update the history candidate list after the history candidate list is updated once; a third acquisition module is used to obtain the history candidate list; an update module is used if the area sum is greater than or equal to a preset threshold, Updating the historical candidate list according to the motion information of the current block.
第七方面，本发明实施例提供一种历史候选列表更新编码器，用于实现上述任一方面的方法和装置。In a seventh aspect, an embodiment of the present invention provides a historical candidate list update encoder, which is used to implement the method and apparatus of any of the foregoing aspects.
第八方面，本发明实施例提供一种历史候选列表更新解码器，用于实现上述任一方面的方法和装置。In an eighth aspect, an embodiment of the present invention provides a historical candidate list update decoder, which is used to implement the method and device of any of the foregoing aspects.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是一种视频编码流程示意图；FIG. 1 is a schematic diagram of a video encoding process;
图2是一种帧间预测示意图；FIG. 2 is a schematic diagram of an inter prediction;
图3是一种视频解码流程示意图；FIG. 3 is a schematic diagram of a video decoding process;
图4是一种运动信息候选位置示意图；4 is a schematic diagram of a candidate position of motion information;
图5是一种构建融合运动信息候选列表的方法；FIG. 5 is a method for constructing a candidate list of fusion motion information; FIG.
图6a是现有技术中一种更新历史候选列表的方法；6a is a method for updating a history candidate list in the prior art;
图6b是一种视频***；Figure 6b is a video system;
图7是本发明中一种更新历史候选列表方法示意图；7 is a schematic diagram of a method for updating a history candidate list in the present invention;
图8是本发明中一种更新历史候选列表方法示意图；8 is a schematic diagram of a method for updating a history candidate list in the present invention;
图9是本发明中一种更新历史候选列表方法示意图；9 is a schematic diagram of a method for updating a history candidate list in the present invention;
图10是本发明中一种更新历史列表装置示意图；FIG. 10 is a schematic diagram of an update history list device in the present invention; FIG.
图11是示出实现内容分发服务的内容供应***3100的示例结构的框图。FIG. 11 is a block diagram showing an example structure of a content supply system 3100 that implements a content distribution service.
图12是示出终端设备的示例的结构的框图。FIG. 12 is a block diagram showing a structure of an example of a terminal device.
具体实施方式detailed description
下面结合本申请实施例中的附图对本申请实施例进行描述。The following describes the embodiments of the present application with reference to the drawings in the embodiments of the present application.
对于编码过程，如图1所示，该过程主要包括帧内预测(Intra Prediction)、帧间预测(Inter Prediction)、变换(Transform)、量化(Quantization)、熵编码(Entropy encode)、环内滤波(in-loop filtering)(主要为去块滤波，de-blocking filtering)等环节。将图像划分为编码块之后进行帧内预测或者帧间预测，并且在得到残差之后进行变换量化，最终进行熵编码并输出码流。此处编码块为由像素点组成的M*N大小的阵列(M可以等于N，也可以不等于N)，并且已知各个像素点位置的像素值。As for the encoding process, as shown in FIG. 1, the process mainly includes intra prediction (Intra Prediction), inter prediction (Inter Prediction), transform (Transform), quantization (Quantization), entropy encoding (Entropy encoding), in-loop filtering (in-loop filtering) (mainly de-blocking filtering). After the image is divided into coding blocks, intra prediction or inter prediction is performed, and after the residuals are obtained, transform quantization is performed, and finally entropy coding is performed and a code stream is output. Here, the coding block is an M * N-sized array composed of pixels (M may be equal to N or not equal to N), and the pixel value of each pixel point position is known.
帧内预测是指利用当前图像内已重建区域内像素点的像素值对当前编码块内像素点的像素值进行预测。Intra prediction refers to using the pixel values of pixels in the reconstructed area in the current image to predict the pixel values of pixels in the current coding block.
帧间预测是在已重建的图像中，为当前图像中的当前编码块寻找匹配的参考块，将参考块中的像素点的像素值作为当前编码块中像素点的像素值的预测信息或者预测值(以下不再区分信息和值)，此过程称为运动估计(Motion estimation，ME)(如图2所示)，并传输当前编码块的运动信息。Inter prediction is to find a matching reference block for the current coding block in the current image in the reconstructed image, and use the pixel value of the pixel point in the reference block as the prediction information or prediction of the pixel value of the pixel point in the current coding block. Value (hereafter information and value are not distinguished), this process is called motion estimation (ME) (as shown in Figure 2), and the motion information of the current coded block is transmitted.
需要说明的是，当前编码块的运动信息包括了预测方向的指示信息(通常为前向预测、后向预测或者双向预测)，一个或两个指向参考块的运动矢量(Motion vector，MV)，以及参考块所在图像的指示信息(通常记为参考帧索引，Reference index)。It should be noted that the motion information of the current coding block includes indication information of the prediction direction (usually forward prediction, backward prediction, or bidirectional prediction), one or two motion vectors (Motion vector, MV) pointing to the reference block, And indication information (usually referred to as a reference frame index) of a picture in which a reference block is located.
前向预测是指当前编码块从前向参考图像集合中选择一个参考图像获取参考块。后向预测是指当前编码块从后向参考图像集合中选择一个参考图像获取参考块。双向预测是指从前向和后向参考图像集合中各选择一个参考图像获取参考块。当使用双向预测方法时，当前编码块会存在两个参考块，每个参考块各自需要运动矢量和参考帧索引进行指示，然后根据两个参考块内像素点的像素值确定当前块内像素点像素值的预测值。Forward prediction means that the current coding block selects a reference image from the forward reference image set to obtain a reference block. Backward prediction means that the current coding block selects a reference image from the backward reference image set to obtain a reference block. Bidirectional prediction refers to selecting a reference image from the forward and backward reference image sets to obtain a reference block. When the bidirectional prediction method is used, there are two reference blocks in the current coding block, each of which requires a motion vector and a reference frame index to indicate, and then the pixels in the current block are determined according to the pixel values of the pixels in the two reference blocks. The predicted value of the pixel value.
运动估计过程需要为当前编码块在参考图像中尝试多个参考块，最终使用哪一个或者哪几个参考块用作预测则使用率失真优化(Rate-distortion optimization，RDO)或者其他方法确定。The motion estimation process needs to try multiple reference blocks in the reference image for the current coding block. Which one or several reference blocks are ultimately used for prediction is determined using rate-distortion optimization (RDO) or other methods.
利用帧内预测或者帧间预测得到预测信息之后，当前编码块内像素点的像素值减去对应的预测信息便得到残差信息，然后利用离散余弦变换(Discrete Cosine Transformation，DCT)等方法对残差信息进行变换，再使用量化熵编码得到码流。预测信号加上重建残差信号之后需进一步进行滤波操作，进而得到重建信号，并将其作为后续编码的参考信号。After obtaining the prediction information by using intra prediction or inter prediction, the pixel information of the pixels in the current coding block is subtracted from the corresponding prediction information to obtain residual information. Then, discrete cosine transform (DCT) and other methods The difference information is transformed, and then the code stream is obtained by using quantized entropy coding. After the prediction signal is added with the reconstructed residual signal, further filtering operations are required to obtain a reconstructed signal and use it as a reference signal for subsequent encoding.
解码则相当于编码的逆过程。例如，如图3所示，首先利用熵解码反量化反变换得到残差信息，解码码流确定当前编码块使用的是帧内预测还是帧间预测。如果是帧内预测，则利用周围已重建区域内像素点的像素值按照所使用的帧内预测方法构建预测信息。如果是帧间预测，则需要解析出运动信息，并使用所解析出的运动信息在已重建的图像中确定参考块，并将块内像素点的像素值作为预测信息，此过程称为运动补偿(Motion compensation，MC)。使用预测信息加上残差信息经过滤波操作便可以得到重建信息。Decoding is equivalent to the reverse process of encoding. For example, as shown in FIG. 3, first, entropy decoding and inverse quantization and inverse transformation are used to obtain residual information. The decoded code stream determines whether the current coding block uses intra prediction or inter prediction. If it is intra prediction, the pixel information of pixels in the surrounding reconstructed area is used to construct prediction information according to the used intra prediction method. If it is inter prediction, you need to parse out the motion information, use the parsed motion information to determine the reference block in the reconstructed image, and use the pixel values of the pixels in the block as prediction information. This process is called motion compensation (Motion Compensation, MC). Reconstruction information can be obtained by using prediction information plus residual information through a filtering operation.
帧间预测是基于运动补偿(motion compensation)的预测技术，如图2所示。在帧间预测编码中，由于图像邻近帧中的相同物体存在一定时域相关性，可将图像序列的每一帧划分成许多互不重叠的块，并认为块内所有像素点的运动都相同。主要处理为确定当前块的运动信息，根据运动信息从参考帧中获取参考图像块，产生当前块的预测图像。运动信息(motion information)包含帧间预测方向、参考帧索引(reference index,ref_idx)、运动矢量(motion vector,MV)等。Inter prediction is a prediction technique based on motion compensation (see Figure 2). In inter-prediction coding, because the same objects in adjacent frames of the image have a certain time-domain correlation, each frame of the image sequence can be divided into many non-overlapping blocks, and all pixels in the block are considered to have the same motion. . The main processing is to determine the motion information of the current block, obtain a reference image block from a reference frame according to the motion information, and generate a predicted image of the current block. Motion information (motion information) includes inter prediction direction, reference frame index (ref_idx), motion vector (MV, etc.).
帧间预测通过帧间预测方向指示当前块使用前向预测、后向预测或双向预测其中的何种预测方向，通过参考帧索引(reference index)指明参考帧(reference frame)，通过运动矢量指示当前块(current block)在参考帧中的参考块(reference block)相对当前帧中当前块的位置偏移。运动矢量指示了参考帧中用于预测当前块的参考图像块相对于当前块的位移矢量，因而一个运动矢量对应一个参考图像块。Inter prediction indicates the prediction direction of the current block using forward prediction, backward prediction, or bidirectional prediction through the inter prediction direction, the reference frame is indicated by the reference index, and the current vector is indicated by the motion vector. The reference block of the current block in the reference frame is offset from the position of the current block in the current frame. The motion vector indicates a displacement vector of a reference image block used to predict the current block relative to the current block in the reference frame, so one motion vector corresponds to one reference image block.
在编码时，H.265/HEVC、H.266/VVC等视频编码标准把一帧图像分割成互不重叠的编码树单元(Coding Tree Unit,CTU)，一个CTU被划分为一个或多个编码单元(Coding Unit,CU)。一个CU包含编码信息，包括预测模式、变换系数等信息。解码端：按照这些编码信息对CU进行相应的预测、反量化、反变换等解码处理，产生这个CU对应的重建图像。During encoding, H.265 / HEVC, H.266 / VVC and other video coding standards divide a frame of image into non-overlapping Coding Tree Units (CTUs). A CTU is divided into one or more codes. Coding Unit (CU). One CU contains coding information, including information such as prediction mode and transform coefficients. Decoding end: Perform corresponding prediction, inverse quantization, inverse transform and other decoding processing on the CU according to the encoded information to generate a reconstructed image corresponding to the CU.
码流中，运动信息占有大量的数据量。为了降低所需数据量，通常采用预测的方式传送运动信息。总体可以分为inter mvp(例如但不限于H.265中的AMVP)和merge两种模式。In the code stream, motion information occupies a large amount of data. In order to reduce the amount of data required, motion information is usually transmitted in a predictive manner. Generally, it can be divided into two modes: intermvp (such as but not limited to AMVP in H.265) and merge.
Inter mvp模式:传送的运动信息包含：帧间预测方向(前向、后向或双向)、参考帧索引、运动矢量预测值索引、运动矢量残差值。对运动信息中的运动矢量信息通常采用传送实际运动矢量与运动矢量预测值(motion vector predictor，MVP)的差值的方式，编码端将MVP与实际运动矢量之间的运动矢量残差值(motion vector difference,MVD)传递到解码端。其中运动矢量预测可能包含多个预测值，一般在编码端和解码端使用相同的方式构建运动矢量预测候选列表(mvp candidate list)，将运动矢量预测值索引(motion vector predictor index,MVP index)传递到解码端；根据运动矢量预测值索引以及运动矢量预测候选列表确定出MVP；再根据MVP和MVD进一步确定出MV。Intermvp mode: The transmitted motion information includes: inter prediction direction (forward, backward, or bidirectional), reference frame index, motion vector prediction value index, and motion vector residual value. For the motion vector information in the motion information, the method of transmitting the difference between the actual motion vector and the motion vector predictor (MVP) is usually used, and the encoder ends the motion vector residual value between the MVP and the actual motion vector (motion vector (difference, MVD) to the decoder. The motion vector prediction may include multiple prediction values. Generally, the motion vector prediction candidate list (mvp candidate list) is constructed in the same way at the encoding end and the decoding end, and the motion vector prediction index (MVP index) is passed. To the decoding end; determine the MVP according to the motion vector prediction value index and the motion vector prediction candidate list; and then further determine the MV according to the MVP and the MVD.
Merge/skip模式：在编码端和解码端使用相同的方式构建融合运动信息候选列表(merge candidate list)，将索引传递到解码端，码流中传送融合索引(merge index)。运动信息候选列表(candidate list)中的运动信息通常从其空域相邻块或参考帧中的时域块中获得，其中由当前块相邻的图像块的运动信息得到的候选运动信息称为空间候选(spatial candidate)，由当前块在参考图像中得到的对应位置图像块的运动信息称为时间候选(temporal candidate)。HEVC以及VTM(Versatile Video Coding Test Model)的当前块空域候选和时域候选如图4所示，其中空间候选包含A0、A1、B0、B1、和B2的运动信息，时域候选包括T0和T1(T0和T1都是参考帧中的)的运动信息。Merge / skip mode: A merge motion candidate list is constructed on the encoding side and the decoding side in the same way, the index is passed to the decoding side, and the merge index is transmitted in the code stream. The motion information in the candidate list of motion information (candidate list) is usually obtained from its spatial domain neighboring blocks or time domain blocks in the reference frame. The candidate motion information obtained from the motion information of the image blocks adjacent to the current block is called space. Candidate. The motion information of the corresponding position image block obtained from the current image in the reference image is called temporal candidate. The current block spatial and time domain candidates of HEVC and VTM (Versatile Video Coding Model) are shown in Figure 4, where the spatial candidates include motion information of A0, A1, B0, B1, and B2, and the time domain candidates include T0 and T1 (T0 and T1 are both in the reference frame).
JVET-K0104提案中提出了在融合运动信息候选列表中加入历史候选(history candidate)的方法，增加了merge/skip模式的融合运动信息候选的数量，提高了预测效率。提案中还提出了在运动矢量预测候选列表(含有运动矢量)中加入历史候选 (history candidate)的方法，增加了Inter MVP模式的运动矢量预测候选的数量，提高了预测效率。历史候选列表是由历史候选组成，其中历史候选为先前编码/解码块(当前片slice中当前块之前已经编码或解码的编码块)的运动信息。可以理解为是一个一直存在的列表，编码或解码完一个图像块，就将该图像块的运动信息添加进去，通常历史候选列表中最多可以包含N个运动信息，N为预设值。JVET-K0104提案中，介绍了历史候选列表(history candidate list)的使用方法以及历史候选列表的构建方法。The JVET-K0104 proposal proposes a method of adding history candidates to the fusion motion information candidate list, which increases the number of fusion motion information candidates in merge / skip mode and improves prediction efficiency. The proposal also proposes a method of adding history candidates to the motion vector prediction candidate list (including motion vectors), which increases the number of motion vector prediction candidates in the InterMVP mode and improves prediction efficiency. The history candidate list is composed of history candidates, where the history candidates are motion information of a previously encoded / decoded block (a coded block that has been encoded or decoded before the current block in the current slice). It can be understood as a list that exists all the time. After encoding or decoding an image block, the motion information of the image block is added. Generally, the historical candidate list can contain at most N motion information, where N is a preset value. The JVET-K0104 proposal introduced the use of history candidate list and the construction method of history candidate list.
加入历史候选的融合运动信息候选列表的构建方法，也可以理解为历史候选列表的使用方法，可以如下：The method of constructing the candidate list of fused motion information added to the historical candidate can also be understood as the method of using the historical candidate list, as follows:
步骤11：将当前块的空间候选和时域候选按照预定规则(例如可以按照HEVC或VTM的添加顺序和查重策略)创建当前块的融合运动信息候选列表list(mi)，其方法与HEVC中的方法相同。如图4所示，空间候选可以包含A0、A1、B0、B1、和B2，时域候选可以包含T0和T1。在具体实现过程中，经过查重，空间候选和时间候选最终放进融合运动信息候选列表的可以为预设数目个候选，如全部或部分。例如，融合运动信息候选列表中先放空域候选，再放时域候选。在VTM中，时域候选也可以包括自适应时域运动矢量预测(ATMVP)技术提供的候选。Step 11: The spatial candidate and the time domain candidate of the current block are created according to predetermined rules (for example, according to the addition order and check strategy of HEVC or VTM), and a list (mi) of the fusion motion information candidate of the current block is created. The method is the same. As shown in FIG. 4, the spatial candidate may include A0, A1, B0, B1, and B2, and the time domain candidate may include T0 and T1. In the specific implementation process, after checking the weight, the spatial candidate and the temporal candidate may be finally put into the fusion motion information candidate list, which may be a preset number of candidates, such as all or part. For example, the airspace candidate is placed first in the fusion motion information candidate list, and then the time domain candidate is placed. In VTM, time-domain candidates may also include candidates provided by the adaptive time-domain motion vector prediction (ATMVP) technology.
步骤12：将历史候选列表list(h)中的历史候选按照预定顺序加入到融合运动信息候选列表中，例如JVET-K0104按照从历史候选列表尾部到头部的顺序检查预设数目的历史候选，如图5所示，从历史候选列表尾部的历史候选开始，检查其与步骤11得到的融合运动信息候选列表中的融合运动信息候选是否相同，如果不同则加入到融合运动信息候选列表中，如预设位置，预设位置可以包含所有空域候选之后，或者放在时域候选之后。如果相同则检查历史候选列表中下一个历史候选，以此类推，直到融合运动信息候选列表中的MV的数量达到一个预设数目，不再进行历史候选的检查和添加了。Step 12: Add historical candidates in the historical candidate list list (h) to the fusion motion information candidate list in a predetermined order. For example, JVET-K0104 checks a preset number of historical candidates in the order from the tail to the head of the history candidate list. As shown in FIG. 5, starting from the historical candidate at the tail of the historical candidate list, check whether it is the same as the fused motion information candidate in the fused motion information candidate list obtained in step 11. Preset position. The preset position may include all spatial domain candidates or be placed after time domain candidates. If they are the same, the next historical candidate in the historical candidate list is checked, and so on, until the number of MVs in the fused motion information candidate list reaches a preset number, and no historical candidate check and addition is performed.
图5：历史候选加入融合运动信息候选列表的示意图Figure 5: Schematic diagram of historical candidates joining the candidate list of fusion motion information
步骤13：作为可选的，若已经遍历完历史候选列表中的所有历史候选，融合运动信息候选列表中的MV的数量还没有达到一个预设数目，则还可以加入其它类型的融合运动信息候选，如双向预测候选(bi-predictive candidate)和零运动矢量候选(zero motion vector candidate)。Step 13: As an option, if all historical candidates in the historical candidate list have been traversed, and the number of MVs in the fusion motion information candidate list has not reached a preset number, other types of fusion motion information candidates may be added. , Such as bi-predictive candidates and zero motion vector candidates.
加入历史候选的运动矢量预测候选列表的构建方式，也可以理解为历史候选列表的使用方法，可以如下：The construction method of the motion vector prediction candidate list added to the historical candidate can also be understood as the use method of the historical candidate list, which can be as follows:
步骤21：将当前块的空间候选和时域候选加入当前块的运动矢量预测候选列表list(mvp)，其方法与HEVC中的方法相同。如图4所示，空间候选包含A0、A1、B0、B1、和B2，时域候选包括T0和T1。Step 21: Add the spatial candidate and the time domain candidate of the current block to the motion vector prediction candidate list (mvp) of the current block, the method of which is the same as the method in HEVC. As shown in FIG. 4, the spatial candidates include A0, A1, B0, B1, and B2, and the time domain candidates include T0 and T1.
步骤22：将历史候选列表list(h)(或者用Hlist来表示)中的历史候选按照加入到运动矢量预测候选列表中，按照从历史候选列表尾部到头部的顺序检查预设数目的历史候选。从历史候选列表尾部的历史候选开始，检查其与步骤21得到的运动矢量预测候选列表中运动矢量预测候选的参考帧索引与历史候选中的运动矢量的参考帧索引是否相同，如果与运动矢量预测候选列表的参考帧索引使用相同参考帧索引的运动矢量则会被加入到运动矢量预测候选列表中，如果不相同则检查历史候选列表中下一 个历史候选，以此类推，直到运动矢量预测候选列表中的MV的数量达到一个预设数目，不再进行历史候选的检查和添加了。Step 22: Add historical candidates in the historical candidate list list (h) (or Hlist) to the motion vector prediction candidate list, and check a preset number of historical candidates in the order from the tail to the head of the history candidate list . Starting from the historical candidate at the end of the historical candidate list, check whether it is the same as the reference frame index of the motion vector prediction candidate in the motion vector prediction candidate list obtained in step 21 and the reference frame index of the motion vector in the historical candidate. Candidate list reference frame index Motion vectors using the same reference frame index will be added to the motion vector prediction candidate list. If they are not the same, the next historical candidate in the historical candidate list will be checked, and so on, until the motion vector prediction candidate list The number of MVs in the game reaches a preset number, and the history candidate check and addition are no longer performed.
步骤23：加入其它类型的运动矢量候选，如零运动矢量候选(zero motion vector candidate)。Step 23: Add other types of motion vector candidates, such as zero motion vector candidates.
JVET-K0104提案中，历史候选列表利用当前帧中已编码块/已解码块的运动信息构建，采用先进先出的方式对于历史候选列表进行访问。编码/解码端中总体的历史候选列表构建方式如下：In the JVET-K0104 proposal, the historical candidate list is constructed using the motion information of the coded / decoded blocks in the current frame, and the historical candidate list is accessed in a first-in-first-out manner. The overall historical candidate list in the encoding / decoding side is constructed as follows:
步骤31：在片(SLICE)解码开始的时候初始化历史候选列表，将其清空。可以理解为，对于每一个新的片，开始解码时都要把历史候选列表初始化清空。Step 31: Initialize the history candidate list at the beginning of slice decoding and clear it. It can be understood that, for each new slice, the history candidate list is initialized and cleared when decoding is started.
步骤32：解码当前CU，如果当前CU或当前块为merge/skip帧间预测模式，可以对应有融合运动信息候选列表，如果当前CU或当前块为inter帧间预测模式，可以对应有运动矢量预测候选列表；同时将历史候选列表中的历史候选加入到融合运动信息候选列表或运动矢量预测候选列表中。Step 32: Decode the current CU. If the current CU or current block is a merge / skip inter prediction mode, it may correspond to a candidate list of fused motion information. If the current CU or current block is an inter inter prediction mode, it may correspond to motion vector prediction. Candidate list; meanwhile, historical candidates in the historical candidate list are added to the fusion motion information candidate list or the motion vector prediction candidate list.
若当前块为merge/skip模式，则根据码流中携带的融合索引和融合运动信息候选列表确定当前块的运动信息。If the current block is in a merge / skip mode, the motion information of the current block is determined according to the fusion index and the fusion motion information candidate list carried in the code stream.
若当前块为Inter MVP模式，则根据码流中传送的帧间预测方向、参考帧索引、运动矢量预测值索引、运动矢量预测候选列表、运动矢量残差值确定当前块的运动信息。If the current block is in the Inter MVP mode, the motion information of the current block is determined according to the inter prediction direction, reference frame index, motion vector prediction value index, motion vector prediction candidate list, and motion vector residual value transmitted in the bitstream.
步骤33：在解码完当前CU或当前块之后，将当前块的运动信息作为新的历史候选，加入到历史候选列表中，进而更新了历史候选列表，如图6a所示。首先，从历史候选列表头部开始，将当前块的运动信息与历史候选列表中历史候选进行比较。如果某个历史候选(例如图6a中的MV2)与当前块的运动信息相同，则将历史列表中的这个历史候选MV2移除，并将当前块的运动信息加进历史候选列表中的尾部；虽然MV2已经被移除，但是相同的运动信息在历史候选列表中的排序位置发生了改变。如果某个历史候选(例如图6a中的MV2)与当前块的运动信息不同，则检查历史候选列表中历史候选的个数，如该列表中历史候选个数超过某个预设值，将该历史候选列表中位于头部的历史候选移除，并将当前块的运动信息加入到历史候选列表的尾部。Step 33: After decoding the current CU or current block, the motion information of the current block is added as a new historical candidate to the historical candidate list, and then the historical candidate list is updated, as shown in FIG. 6a. First, starting from the head of the historical candidate list, the motion information of the current block is compared with historical candidates in the historical candidate list. If a historical candidate (for example, MV2 in FIG. 6a) is the same as the motion information of the current block, remove the historical candidate MV2 from the history list and add the motion information of the current block to the tail of the historical candidate list; Although MV2 has been removed, the ranking position of the same motion information in the historical candidate list has changed. If a historical candidate (such as MV2 in Figure 6a) is different from the motion information of the current block, check the number of historical candidates in the historical candidate list. If the number of historical candidates in the list exceeds a preset value, change the The historical candidate at the head of the historical candidate list is removed, and the motion information of the current block is added to the tail of the historical candidate list.
对一个slice中的每一个块，执行步骤32-步骤33，直到slice中的所有的块全部遍历完成；对下一个slice，再去执行步骤31-33。For each block in a slice, perform steps 32-33 until all the blocks in the slice have been traversed; for the next slice, go to steps 31-33.
在一种可能的实现过程中，以一个slice编码或解码开始对历史候选列表的构建和使用进行举例，假设更新的过程中没有相同的运动信息：In a possible implementation process, a slice encoding or decoding is used to start the construction and use of the historical candidate list. Assume that the same motion information is not available during the update process:
对于block1，步骤31使用：Hlist(NULL)；……；步骤34构建：Hlist(V1)For block1, step 31 uses: Hlist (NULL); ...; step 34 builds: Hlist (V1)
对于block2，步骤31使用：Hlist(V1)；……；步骤34构建：Hlist(V1，V2)For block2, step 31 uses: Hlist (V1); ...; step 34 builds: Hlist (V1, V2)
对于block3，步骤31使用：Hlist(V1，V2)；……；步骤34构建：Hlist(V1，V2，V3)……；For block3, step 31 uses: Hlist (V1, V2); ...; step 34 builds: Hlist (V1, V2, V3) ...;
对于block7，步骤31使用：Hlist(V1，V2，……，V6)；……；步骤34构建：Hlist(V1，V2，……，V7)；For block7, step 31 uses: Hlist (V1, V2, ..., V6); ...; step 34 builds: Hlist (V1, V2, ..., V7);
对于block8，步骤31使用：Hlist(V1，V2，……，V7)；……；步骤34构建：Hlist(V1，V2，……，V8)；For block8, step 31 uses: Hlist (V1, V2, ..., V7); ...; Step 34 builds: Hlist (V1, V2, ..., V8);
假设历史候选列表中最多能够容纳的历史候选为8个，则“去头填尾”，如，Assume that there are 8 historical candidates that can be accommodated in the historical candidate list.
对于block9，步骤31使用：Hlist(V1，V2，……，V8)；……；步骤34构建：Hlist(V2，V3，……，V9)；For block9, step 31 uses: Hlist (V1, V2, ..., V8); ...; Step 34 builds: Hlist (V2, V3, ..., V9);
……...
依次类推。此外，如果构建列表时，发现重复的历史候选，则将原历史候选列表中的历史候选删除，并将当前块的历史候选加进历史候选列表中的尾部。And so on. In addition, if a duplicate historical candidate is found when the list is constructed, the historical candidate in the original historical candidate list is deleted, and the historical candidate of the current block is added to the tail of the historical candidate list.
然而针对上述方法，对每一个编码块在编解码完成后，都要进行历史候选列表格更新，尤其历史候选列表较长时，其构造和更新的时间消耗会较长。而且该方法在使用历史候选列表(假设长度为K，即最多包含K个历史候选)构造融合运动信息候选列表(假设长度为L，即最多包含L个融合运动信息候选)或运动矢量预测候选列表(假设长度为J，即最多包含L个运动矢量预测候选)时，后续的操作分别需要K*J*L次重复项查找，造成程序执行时间的大幅增长。However, according to the above method, after encoding and decoding is completed for each coding block, the historical candidate list grid is updated. Especially, when the historical candidate list is long, the time consumption of constructing and updating it will be longer. Moreover, the method uses a history candidate list (assuming length K, that is, contains a maximum of K historical candidates) to construct a fusion motion information candidate list (assuming length L, that is, contains a maximum of L fusion motion information candidates) or a motion vector prediction candidate list (Assuming that the length is J, that is, including at most L motion vector prediction candidates), the subsequent operations respectively require K * J * L duplicate item searches, resulting in a substantial increase in program execution time.
本发明提出一种改进的历史候选列表更新和使用的方法，在特定条件下才进行历史候选列表格更新。也使得在使用历史候选列表构造融合运动信息候选列表或运动矢量预测候选列表时，能够跳过某些历史候选。减少复杂度。The present invention proposes an improved method for updating and using a history candidate list, and the history candidate list is updated only under specific conditions. It also enables some historical candidates to be skipped when constructing a fused motion information candidate list or a motion vector prediction candidate list using a historical candidate list. Reduce complexity.
本发明的***框架如图6b所示。本发明在***框架中主要位于视频编码和视频解码中。现有视频传输***通常由采集、编码、发送、接收、解码、和显示这些部分组成。采集模块包含摄像头或摄像头组以及前处理，将光信号转化为数字化的视频序列。接着视频序列经编码器编码，转化为码流。然后码流由发送模块经网络发送至接收模块，经接收模块转化为码流后经解码器解码重建为视频序列。最后重建视频序列经渲染等后处理送至显示设备显示。The system framework of the present invention is shown in Figure 6b. The present invention is mainly located in the video encoding and video decoding in the system framework. Existing video transmission systems usually consist of these parts: capture, encode, send, receive, decode, and display. The acquisition module contains a camera or camera group and pre-processing to convert the light signal into a digitized video sequence. The video sequence is then encoded by an encoder and converted into a code stream. The code stream is then sent by the sending module to the receiving module via the network, and converted into a code stream by the receiving module, and then decoded and reconstructed by the decoder into a video sequence. Finally, the reconstructed video sequence is sent to a display device for display after post-processing such as rendering.
现有视频传输***通常由采集、编码、发送、接收、解码、和显示这些部分组成。采集模块包含摄像头或摄像头组以及前处理，将光信号转化为数字化的视频序列。接着视频序列经编码器编码，转化为码流。然后码流由发送模块经网络发送至接收模块，经接收模块转化为码流后经解码器解码重建为视频序列。最后重建视频序列经渲染等后处理送至显示设备显示。本发明在***框架中主要位于视频编码和视频解码中。Existing video transmission systems usually consist of these parts: capture, encode, send, receive, decode, and display. The acquisition module contains a camera or camera group and pre-processing to convert the light signal into a digitized video sequence. The video sequence is then encoded by an encoder and converted into a code stream. The code stream is then sent by the sending module to the receiving module via the network, and converted into a code stream by the receiving module, and then decoded and reconstructed by the decoder into a video sequence. Finally, the reconstructed video sequence is sent to a display device for display after post-processing such as rendering. The present invention is mainly located in the video encoding and video decoding in the system framework.
本发明的应用场景如图1和图3的基于混合编码框架的视频编码和解码***所示。对于编码过程，如图1所示，主要包括帧内预测(Intra Prediction)、帧间预测(Inter Prediction)、变换(Transform)、量化(Quantization)、熵编码(Entropy encode)、环路内滤波(Loop filter)等环节，分别完成从当前帧临近像素得到预测块、计算MV信息并从参考帧取得预测块、将残差从像素域变换至变换域、对变换域系数进行压缩、对于编码信息进行压缩、对于重建图像进行后处理等作用。对于解码***，如图3所示，相当于编码的逆过程。本发明在视频编码或解码***中主要应用于帧间预测，它应用于帧间预测模式的运动矢量预测中，在编码端和解码端更新和使用历史候选列表的处理是相同的。下面实施例以解码端为例展开描述，对于编码端，本领域技术人员通过解码端的方法和装置描述理应能够复现出来相关的编码方法，本申请中对编码端不予以赘述。An application scenario of the present invention is shown in a video coding and decoding system based on a hybrid coding framework shown in FIGS. 1 and 3. For the encoding process, as shown in Figure 1, it mainly includes intra prediction (Intra Prediction), inter prediction (Inter Prediction), transform (Transform), quantization (Quantization), entropy encoding (Entropy encoding), and in-loop filtering ( Loop filter) and other steps, respectively complete the prediction block from the neighboring pixels of the current frame, calculate the MV information and obtain the prediction block from the reference frame, transform the residual from the pixel domain to the transform domain, compress the transform domain coefficients, and perform the encoding information Compression, post-processing of the reconstructed image, etc. For the decoding system, as shown in Figure 3, it is equivalent to the inverse process of encoding. The present invention is mainly applied to inter prediction in a video encoding or decoding system. It is applied to motion vector prediction in an inter prediction mode, and the processing of updating and using the historical candidate list is the same at the encoding end and the decoding end. The following embodiment uses the decoder as an example to describe the description. For the encoder, those skilled in the art should be able to reproduce the relevant encoding method through the method and device description of the decoder. The encoder will not be described in detail in this application.
本发明涉及一种历史候选列表更新的方法，以及使用更新后历史候选列表中的历 史候选对图像块进行解码的处理。历史候选列表的长度K为预设值，即构造完成后历史候选列表中包含的历史候选运动信息候选个数为K，K为大于0的正整数。The present invention relates to a method for updating a history candidate list, and a process of decoding an image block using the history candidate in the updated history candidate list. The length K of the historical candidate list is a preset value, that is, the number of historical candidate motion information candidates included in the historical candidate list after construction is K, and K is a positive integer greater than 0.
历史候选列表的初始化过程可以为现有技术，该过程可采用与JVET-K0104提案相同的方法进行，即在片SLICE开始解码的时候，将历史候选列表清空，也可采用其他历史候选列表的初始化方法，本发明不作限定。The history candidate list initialization process can be the existing technology. This process can be performed in the same way as the JVET-K0104 proposal, that is, when the SLICE starts decoding, the history candidate list is emptied, or other history candidate list initialization can be used. The method is not limited by the present invention.
对图像中的至少一个使用帧间预测的图像块进行解码处理，得到该图像块的重建图像。上述解码处理包括步骤41至步骤43，正在执行解码处理的图像块称为当前块。Decode at least one image block in the image using inter prediction to obtain a reconstructed image of the image block. The above-mentioned decoding process includes steps 41 to 43, and an image block being decoded is referred to as a current block.
步骤41：解析当前块的帧间预测模式，若当前块为merge/skip模式，则生成融合运动信息候选列表。若当前CU或当前块为Inter MVP模式，则生成运动矢量预测候选列表。根据预设规则将历史候选列表中的历史候选加入到融合运动信息候选列表或运动矢量预测候选列表。Step 41: Parse the inter prediction mode of the current block. If the current block is in merge / skip mode, generate a candidate list of fused motion information. If the current CU or current block is in the Inter MVP mode, a motion vector prediction candidate list is generated. Historical candidates in the historical candidate list are added to the fused motion information candidate list or the motion vector prediction candidate list according to a preset rule.
在具体实现过程中，若当前块的帧间预测模式为merge/skip模式，生成融合运动信息候选列表的方法可以采用上述步骤11-步骤13。In a specific implementation process, if the inter prediction mode of the current block is a merge / skip mode, the method for generating a candidate list of fused motion information may adopt the above steps 11 to 13.
例如，该过程可采用HEVC或者VTM中的方法进行，也可采用其他生成融合运动信息候选列表的方法。若将历史候选列表中的历史候选按照加入到融合运动信息候选列表中，则可以按照JVET-K0104提案中的方法，从历史候选列表尾部到头部的顺序检查并添加预设数目的历史候选。从历史候选列表尾部的历史候选开始，检查其与步骤11得到的融合运动信息候选列表中融合运动信息候选是否相同，如果不同则加入到融合运动信息候选列表中，如果相同则检查历史候选列表中下一个历史候选。需要说明的是，在历史候选加入到融合运动信息候选列表之后，还可以加入其它类型的融合候选，如双向预测融合候选(bi-predictive merge candidate)或零运动矢量融合候选(zero motion vector merge candidate)。For example, the process may be performed using a method in HEVC or VTM, or other methods for generating a candidate list of fused motion information. If historical candidates in the historical candidate list are added to the fusion motion information candidate list in accordance with the method in the JVET-K0104 proposal, a preset number of historical candidates can be checked and added in the order from the tail to the head of the history candidate list. Starting from the historical candidate at the end of the historical candidate list, check whether it is the same as the fused motion information candidate in the fused motion information candidate list obtained in step 11. If they are different, add them to the fused motion information candidate list. If they are the same, check the historical candidate list Next historical candidate. It should be noted that after the historical candidate is added to the fusion motion information candidate list, other types of fusion candidates can be added, such as bi-predictive merge candidate or zero motion vector fusion candidate. ).
在具体实现过程中，若当前块的帧间预测模式为Inter MVP模式，生成运动矢量预测候选列表的方法可以采用步骤21-步骤23。In a specific implementation process, if the inter prediction mode of the current block is the Inter MVP mode, the method for generating a motion vector prediction candidate list may adopt steps 21 to 23.
例如，可采用HEVC或者VTM中的方法进行，也可采用其他生成运动矢量预测候选列表的方法，本发明不作限定。若将历史候选列表中的历史候选加入到运动矢量预测候选列表，可以按照JVET-K0104提案中的方法。从历史候选列表尾部到头部的顺序检查并添加预设数目的历史候选，只有与MVP模式目标参考帧索引使用相同参考帧索引的历史候选才会被加入到运动矢量预测候选列表中。历史候选被加入到运动矢量预测候选列表中时域运动矢量预测候选后，本发明中不予以限定。For example, the method in HEVC or VTM may be adopted, and other methods for generating a motion vector prediction candidate list may also be adopted, which is not limited in the present invention. If the historical candidates in the historical candidate list are added to the motion vector prediction candidate list, the method in the JVET-K0104 proposal can be followed. Check and add a preset number of historical candidates in order from the tail to the head of the historical candidate list. Only historical candidates that use the same reference frame index as the MVP mode target reference frame index will be added to the motion vector prediction candidate list. After the historical candidate is added to the motion vector prediction candidate list in the motion vector prediction candidate list, it is not limited in the present invention.
在具体实现过程中，如拉通多个解码块来看，一些解码块可以采用步骤11-步骤13，另一些解码块可以采用上述步骤21-步骤23，还可以有一些解码块既不构建运动矢量候选列表，也不构建融合运动信息候选列表，可以根据预设规则以及***性能进行配置。In the specific implementation process, such as pulling through multiple decoding blocks, some decoding blocks can use steps 11 to 13 and other decoding blocks can use the above steps 21 to 23, and there may be some decoding blocks that do not build motion. The vector candidate list does not build a candidate list of fused motion information, and can be configured according to preset rules and system performance.
步骤42：获取当前块的运动信息。Step 42: Acquire the motion information of the current block.
更具体的，对于解码端，若当前块为merge/skip模式，则根据码流中携带的融合索引和融合运动信息候选列表确定当前块的运动信息。More specifically, for the decoding end, if the current block is in a merge / skip mode, the motion information of the current block is determined according to the fusion index and the fusion motion information candidate list carried in the code stream.
若当前块为Inter MVP模式，则根据码流中传送的帧间预测方向、参考帧索引、运动矢量预测值索引、运动矢量预测候选列表、运动矢量残差值确定当前块运动信息。If the current block is in the Inter MVP mode, the current block motion information is determined according to the inter prediction direction, reference frame index, motion vector prediction value index, motion vector prediction candidate list, and motion vector residual value transmitted in the bitstream.
步骤43：通过判定一些条件，决定是否根据当前块的运动信息对历史候选列表进行更新。可以实现的方式至少包括步骤43a、步骤43b、步骤43c三种并列的实现方式(请分别参照图7、8、9)。Step 43: Determine whether to update the historical candidate list according to the motion information of the current block by determining some conditions. The achievable manner includes at least three parallel implementation manners of step 43a, step 43b, and step 43c (please refer to Figs. 7, 8, and 9 respectively).
步骤43a：可以包括步骤43a1、43a2和步骤43a3三个步骤。Step 43a: Steps 43a1, 43a2 and 43a3 may be included.
43a1：从解码器得到当前块的深度值(depth)；43a1: Obtain the depth value (depth) of the current block from the decoder;
43a2：获取历史候选列表；43a2: Get historical candidate list;
43a3：如果深度值大于某个预设值H，则不以当前块的运动信息对历史候选列表进行更新，该预设值H为大于0的正整数，且小于编码器或解码器允许的最大深度值。如果深度值小于等于某个预设值，则可以按照JVET-K0104提案中的方法，也可采用其他方法根据当前块的运动信息对历史候选列表进行更新。JVET-K0104提案中，从历史候选列表头部开始，将当前块的运动信息与历史候选列表中的历史候选进行比较；如有某个历史候选与当前块运动信息相同，将该历史候选从历史候选列表中移除。然后，检查历史候选列表大小，如该列表大小超过预设的大小，则将该列表中位于头部的历史候选移除。最后，将当前块的运动信息加入到历史候选列表中。43a3: If the depth value is greater than a preset value H, the historical candidate list is not updated with the motion information of the current block. The preset value H is a positive integer greater than 0 and is less than the maximum allowed by the encoder or decoder. Depth value. If the depth value is less than or equal to a preset value, the historical candidate list may be updated according to the method of the JVET-K0104 proposal or other methods according to the motion information of the current block. In the JVET-K0104 proposal, starting from the head of the historical candidate list, the motion information of the current block is compared with the historical candidate in the historical candidate list; if a historical candidate is the same as the current block motion information, the historical candidate is removed from the history. Removed from the candidate list. Then, check the history candidate list size. If the list size exceeds a preset size, remove the historical candidate at the head of the list. Finally, the motion information of the current block is added to the historical candidate list.
发明方案相比于现有技术，在CU级别根据当前块的深度值对于历史候选列表进行选择性更新。Compared to the prior art, the historical candidate list is selectively updated at the CU level according to the depth value of the current block.
步骤43b：可以包括步骤43b1、43b2和步骤43b3三个步骤。Step 43b: It may include three steps of steps 43b1, 43b2, and 43b3.
43b1：根据解码器得到的当前块中预设位置的点的坐标(x，y)；也可以等同获取块的位置。43b1: According to the coordinates (x, y) of a point at a preset position in the current block obtained by the decoder; the position of the block may also be obtained equivalently.
43b2：获取历史候选列表；43b2: Get historical candidate list;
43b3：如果x不是某个预设值I的整数倍，或者y不是某个预设值P的整数倍，则不以当前块的运动信息对历史候选列表进行更新，该预设值I和P为大于1的正整数。预设位置可以是当前块地左上角顶点(相对坐标偏移为(0,0))，也可以是当前块的中心点(相对坐标偏移为(当前块宽度除以二，当前块高度除以二))。如果x是某个预设值I的整数倍，且y是某个预设值P的整数倍，可以按照JVET-K0104提案中的方法，也可采用其他方法根据当前块的运动信息对历史候选列表进行更新。JVET-K0104提案中，从历史候选列表头部开始，将当前块的运动信息与历史候选列表中的历史候选进行比较；如有某个历史候选与当前块运动信息相同，将该历史候选从历史候选列表中移除。然后，检查历史候选列表大小，如该列表大小超过预设的大小，则将该列表中位于头部的历史候选移除。最后，将当前块的运动信息加入到历史候选列表中。43b3: If x is not an integer multiple of a preset value I, or y is not an integer multiple of a preset value P, the historical candidate list is not updated with the motion information of the current block, the preset values I and P Is a positive integer greater than 1. The preset position can be the top left corner of the current block (relative coordinate offset is (0,0)), or it can be the center point of the current block (relative coordinate offset is (the current block width divided by two, the current block height divided by To two)). If x is an integer multiple of a preset value I and y is an integer multiple of a preset value P, the method in the JVET-K0104 proposal can be used, or other methods can be used to historical candidates based on the motion information of the current block. The list is updated. In the JVET-K0104 proposal, starting from the head of the historical candidate list, the motion information of the current block is compared with the historical candidate in the historical candidate list; if a historical candidate is the same as the current block motion information, the historical candidate is removed from the history. Removed from the candidate list. Then, check the history candidate list size. If the list size exceeds a preset size, remove the historical candidate at the head of the list. Finally, the motion information of the current block is added to the historical candidate list.
发明方案相比于现有技术，在CU级别根据当前块中预设位置的点的坐标对于历史候选列表进行选择性更新。Compared with the prior art, the historical candidate list is selectively updated at the CU level according to the coordinates of a point at a preset position in the current block.
步骤43c：可以包括步骤43c1、43c2和步骤43c3三个步骤。Step 43c: It may include three steps of steps 43c1, 43c2, and 43c3.
43c1：将累计未更新编码块的面积初始化为0，累计未更新编码块面积S为在同一片中从上次更新所在的编码块开始到当前块，中间累计的未进行历史候选列表更新的编码或解码块的总面积；即当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和。若一个编码块的运动信息未用于进行历史候选列表更新，则将其面积加入到累计未更新编码块面积S中。43c1: Initialize the area of the cumulative unupdated coding block to 0, and the area of the cumulative unupdated coding block S is the code from the coding block where the last update is located to the current block in the same slice, and the coding of the historical candidate list is not performed in the middle. Or the total area of the decoded block; that is, the total area of all decoded blocks in the slice where the current block is not used to update the historical candidate list since the last update of the historical candidate list. If the motion information of a coding block is not used to update the historical candidate list, its area is added to the cumulative unupdated coding block area S.
43c2：获取历史候选列表；43c2: Get historical candidate list;
43c3：如果累计未更新编码块面积S小于预设值，则将当前块的面积加入到累计未更新编码块面积，该预设值为大于0的正整数。若累计未更新编码块面积S大于等于该预设值，则将累计未更新编码块面积清0，并可以按照JVET-K0104提案中的方法，也可采用其他方法利用当前块的运动信息对历史候选列表进行更新。JVET-K0104提案中，从历史候选列表头部开始，将当前块的运动信息与历史候选列表中的历史候选进行比较；如有某个历史候选与当前块运动信息相同，将该历史候选从历史候选列表中移除。然后，检查历史候选列表大小，如该列表大小超过预设的大小，则将该列表中位于头部的历史候选移除。最后将当前块的运动信息加入到历史候选列表中。43c3: If the cumulative unupdated coding block area S is smaller than a preset value, the area of the current block is added to the cumulative unupdated coding block area, and the preset value is a positive integer greater than 0. If the cumulative unupdated coding block area S is greater than or equal to the preset value, the cumulative unupdated coding block area is cleared to 0, and the method in the JVET-K0104 proposal may be used, or other methods may be used to use the motion information of the current block to history The candidate list is updated. In the JVET-K0104 proposal, starting from the head of the historical candidate list, the motion information of the current block is compared with the historical candidate in the historical candidate list; if a historical candidate is the same as the current block motion information, the historical candidate is removed from the history. Removed from the candidate list. Then, check the history candidate list size. If the list size exceeds a preset size, remove the historical candidate at the head of the list. Finally, the motion information of the current block is added to the historical candidate list.
相比于现有技术，在CU级根据累计未更新编码块面积选择是否对于历史候选列表进行更新。Compared with the prior art, at the CU level, whether to update the historical candidate list is selected according to the cumulative unupdated coding block area.
此外，在步骤42之后，方法还可以包括步骤44。In addition, after step 42, the method may further include step 44.
步骤44：根据运动信息得到当前块的帧间预测图像，将帧间预测图像与残差图像相加，得到当前块的重建图像。Step 44: Obtain an inter prediction image of the current block according to the motion information, and add the inter prediction image and the residual image to obtain a reconstructed image of the current block.
应理解，在当前块执行步骤43之后，若当前块的运动信息没有对历史候选列表进行更新，则当前的历史候选列表依旧作为下一个解码块的历史候选列表，执行步骤41和步骤42。若当前块的运动信息有对历史候选列表进行更新，则新的历史候选列表将作为下一个解码块的历史候选列表，执行步骤41和步骤42。以此类推，此处不再赘述。It should be understood that after step 43 is performed on the current block, if the motion information of the current block does not update the historical candidate list, the current historical candidate list is still used as the historical candidate list of the next decoding block, and steps 41 and 42 are performed. If the historical candidate list is updated with the motion information of the current block, the new historical candidate list will be used as the historical candidate list of the next decoded block, and steps 41 and 42 are performed. The rest can be deduced by analogy.
在本发明中，并不限定判断当前块运动信息与历史候选列表中某个历史候选是否相同的方法。可以是两个运动信息完全相同，也可以是两个运动信息在某种处理后相同，例如两个运动矢量右移2位后的结果相同。In the present invention, the method for determining whether the current block motion information is the same as a certain historical candidate in the historical candidate list is not limited. It can be that the two pieces of motion information are completely the same, or that the two pieces of motion information are the same after some processing, for example, the results of two motion vectors shifted to the right by 2 bits are the same.
通过本发明提供的更新历史列表的方法，历史候选列表加入融合运动信息候选列表或运动矢量预测候选列表中时，有选择的跳过部分历史候选，同时减少历史候选列表加入融合运动信息候选列表或运动矢量预测候选列表中时重复项检查操作的数目。在具有相当的编码压缩效率的前提下，降低编解码时间。可以在不增加额外存储区域和具有相当的编码效率的前提下，减少了历史候选列表的更新次数，提高了编码效率。With the method for updating the history list provided by the present invention, when the historical candidate list is added to the fusion motion information candidate list or the motion vector prediction candidate list, part of the historical candidates are selectively skipped, and the historical candidate list is reduced to be added to the fusion motion information candidate list or The number of duplicate check operations when motion vector prediction is in the candidate list. Under the premise of considerable coding and compression efficiency, the codec time is reduced. It can reduce the number of updates of the historical candidate list and increase the coding efficiency without adding additional storage areas and having a comparable coding efficiency.
可选的，本申请实施例提供一种历史候选列表更新装置，可如图10所示，所述装置包括：Optionally, an embodiment of the present application provides a device for updating a historical candidate list. As shown in FIG. 10, the device includes:
第一获取模块501，用于获取当前块的运动信息；A first acquisition module 501, configured to acquire motion information of a current block;
第二获取模块502，用于获取当前块的深度信息；A second acquisition module 502, configured to acquire depth information of a current block;
第三获取模块503，用于获取历史候选列表；A third acquisition module 503, configured to acquire a history candidate list;
更新模块504，用于若所述当前块的深度信息小于或等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the depth information of the current block is less than or equal to a preset threshold.
可选的，本申请实施例提供一种历史候选列表更新装置，可如图10所示，所述装置包括：Optionally, an embodiment of the present application provides a device for updating a historical candidate list. As shown in FIG. 10, the device includes:
第一获取模块501，用于获取当前块的运动信息；A first acquisition module 501, configured to acquire motion information of a current block;
第二获取模块502，用于获取当前块的位置；A second obtaining module 502, configured to obtain a position of a current block;
第三获取模块503，用于获取历史候选列表；A third acquisition module 503, configured to acquire a history candidate list;
更新模块504，用于若所述当前块的预设位置位于预设像素的整数倍，则根据所述当前块的运动信息对所述历史候选列表进行更新；其中，所述当前块的预设位置包括当前块的左上顶点、上边界中点、中心点、或左边界中心点。An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the preset position of the current block is an integer multiple of a preset pixel; wherein the preset of the current block is The position includes the upper left vertex, the upper boundary midpoint, the center point, or the left boundary center point of the current block.
可选的，本申请实施例提供一种历史候选列表更新装置，可如图10所示，所述装置包括：Optionally, an embodiment of the present application provides a device for updating a historical candidate list. As shown in FIG. 10, the device includes:
第一获取模块501，用于获取当前块的运动信息；A first acquisition module 501, configured to acquire motion information of a current block;
第二获取模块502，用于获取当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和；A second obtaining module 502, configured to obtain the total area of all decoded blocks in the slice where the current block is not used to update the historical candidate list since the historical candidate list was last updated;
第三获取模块503，用于获取历史候选列表；A third acquisition module 503, configured to acquire a history candidate list;
更新模块504，用于若所述面积总和大于等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。An update module 504 is configured to update the historical candidate list according to the motion information of the current block if the total area is greater than or equal to a preset threshold.
具体地，第一获取模块501用于执行步骤41和步骤42相应的方法以及可以对其进行部分等同替换的方法；第二获取模块502用于执行步骤43a1或43b1或43c1相应的方法以及可以对其进行部分等同替换的方法；第三获取模块用于执行步骤43a2或43b2或43c2相应的方法以及可以对其进行部分等同替换的方法；更新模块505用于执行步骤43a3或43b3或43c3相应的方法以及可以对其进行部分等同替换的方法。Specifically, the first acquisition module 501 is configured to execute the methods corresponding to steps 41 and 42 and a method that can be partially equivalently replaced; the second acquisition module 502 is configured to execute the methods corresponding to step 43a1 or 43b1 or 43c1 and can be applied to A method for performing partial equivalent replacement thereof; a third obtaining module for performing a method corresponding to step 43a2 or 43b2 or 43c2 and a method capable of performing partial equivalent replacement thereof; an updating module 505 for performing a method corresponding to step 43a3 or 43b3 or 43c3 And methods that can be partially replaced equivalently.
本申请实施例提供一种编码设备，包括：相互耦合的非易失性存储器和处理器，所述处理器调用存储在所述存储器中的程序代码以执行步骤41、步骤42以及步骤43a、或步骤43b或步骤43c的方法以及等同方法。An embodiment of the present application provides a coding device, which includes a non-volatile memory and a processor coupled to each other, and the processor calls program code stored in the memory to perform step 41, step 42, and step 43a, or The method of step 43b or step 43c and the equivalent method.
本申请实施例提供一种解码设备，包括：相互耦合的非易失性存储器和处理器，所述处理器调用存储在所述存储器中的程序代码以执行步骤41、步骤42以及步骤43a、或步骤43b或步骤43c的方法以及等同方法。An embodiment of the present application provides a decoding device, including: a non-volatile memory and a processor coupled to each other, the processor calling program code stored in the memory to perform step 41, step 42, and step 43a, or The method of step 43b or step 43c and the equivalent method.
本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储了程序代码，其中，所述程序代码包括用于执行步骤41、步骤42以及步骤43a、或步骤43b或步骤43c的方法以及等同方法中的部分或全部步骤的指令。An embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores program code, where the program code includes steps for performing step 41, step 42, and step 43a, or step 43b or step 43c. Instructions and some or all of the steps in the equivalent method.
本申请实施例提供一种计算机程序产品，当所述计算机程序产品在计算机上运行时，使得所述计算机执行步骤41、步骤42以及步骤43a、或步骤43b或步骤43c的方法以及等同方法中的部分或全部步骤。An embodiment of the present application provides a computer program product, and when the computer program product runs on a computer, the computer causes the computer to execute the method in step 41, step 42, and step 43a, or step 43b or step 43c and equivalent Some or all steps.
应理解，方法实施例中一些技术特征的解释和表述同样适用在装置实施例、编码设备、解码设备、计算机程序、存储介质等。It should be understood that the explanation and expression of some technical features in the method embodiments are also applicable to the device embodiments, the encoding device, the decoding device, the computer program, the storage medium, and the like.
下面说明上述实施例中所示的编码方法以及解码方法的应用，以及使用所述编码方法和解码方法的***。The following describes applications of the encoding method and the decoding method shown in the above embodiments, and a system using the encoding method and the decoding method.
图11是示出用于实现内容分发服务的内容供应***3100的框图。此内容供应***3100包括捕获设备3102、终端设备3106，并且可选地包括显示器3126。捕获设备3102通过通信链路3104与终端设备3106通信。通信链路可以包括上文描述的通信信道13。通信链路3104包括但不限于WIFI、以太网、有线、无线(3G/4G/5G)、USB或其任何 种类的组合等。FIG. 11 is a block diagram showing a content supply system 3100 for implementing a content distribution service. This content provisioning system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 through a communication link 3104. The communication link may include the communication channel 13 described above. The communication link 3104 includes, but is not limited to, WIFI, Ethernet, wired, wireless (3G / 4G / 5G), USB, or any combination thereof.
捕获设备3102生成数据，并可以通过如上述实施例中所示的编码方法对数据进行编码。可替换地，捕获设备3102可以将数据分发到流媒体服务器(图11中未示出)，服务器对数据进行编码并将编码数据传输到终端设备3106。捕获设备3102包括但不限于相机、智能手机或平板电脑、计算机或笔记本电脑、视频会议***、PDA、车载设备，或其中任何一个的组合等。例如，捕获设备3102可以包括如上文所述的源设备12。当数据包括视频时，包括在捕获设备3102中的视频编码器20可以实际执行视频编码处理。当数据包括音频(即，语音)时，包括在捕获设备3102中的音频编码器可以实际执行音频编码处理。对于一些实际场景，捕获设备3102通过将编码视频数据和编码音频数据一起复用来分发编码视频数据和编码音频数据。对于其它实际场景，例如在视频会议***中，不复用编码音频数据和编码视频数据。捕获设备3102分别将编码音频数据和编码视频数据分发到终端设备3106。The capture device 3102 generates data, and can encode the data by an encoding method as shown in the above embodiments. Alternatively, the capture device 3102 may distribute the data to a streaming media server (not shown in FIG. 11), the server encodes the data and transmits the encoded data to the terminal device 3106. The capturing device 3102 includes, but is not limited to, a camera, a smartphone or a tablet computer, a computer or a notebook computer, a video conference system, a PDA, a vehicle-mounted device, or a combination of any of them. For example, the capture device 3102 may include the source device 12 as described above. When the data includes video, the video encoder 20 included in the capture device 3102 may actually perform a video encoding process. When the data includes audio (ie, speech), the audio encoder included in the capture device 3102 can actually perform audio encoding processing. For some practical scenarios, the capture device 3102 distributes the encoded video data and the encoded audio data by multiplexing the encoded video data and the encoded audio data together. For other practical scenarios, such as in a video conference system, encoded audio data and encoded video data are not multiplexed. The capture device 3102 distributes the encoded audio data and the encoded video data to the terminal device 3106, respectively.
在内容供应***3100中，终端设备310接收并再现编码数据。终端设备3106可以是具有数据接收和恢复能力的设备，例如智能手机或平板电脑3108、计算机或笔记本电脑3110、网络视频录像机(network video recorder，NVR)/数字视频录像机(digital video recorder，DVR)3112、TV 3114、机顶盒(set top box，STB)3116、视频会议***3118、视频监控***3120、个人数字助理(personal digital assistant，PDA)3122、车载设备3124，或能够解码上述编码数据的以上设备中任何一个的组合等。例如，终端设备3106可以包括如上所述的目的地设备14。当编码数据包括视频时，包括在终端设备中的视频解码器30被优先化以执行视频解码。当编码数据包括音频时，包括在终端设备中的音频解码器被优先化以执行音频解码处理。In the content supply system 3100, the terminal device 310 receives and reproduces the encoded data. The terminal device 3106 can be a device with data receiving and recovery capabilities, such as a smart phone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112 , TV 3114, set-top box (STB) 3116, video conference system 3118, video surveillance system 3120, personal digital assistant (PDA) 3122, in-vehicle equipment 3124, or above devices capable of decoding the above-mentioned encoded data Any combination of them etc. For example, the terminal device 3106 may include the destination device 14 as described above. When the encoded data includes a video, the video decoder 30 included in the terminal device is prioritized to perform video decoding. When the encoded data includes audio, the audio decoder included in the terminal device is prioritized to perform audio decoding processing.
对于具有其显示器的终端设备，例如，智能手机或平板电脑3108、计算机或笔记本电脑3110、网络视频录像机(network video recorder，NVR)/数字视频录像机(digital video recorder，DVR)3112、TV 3114、个人数字助理(personal digital assistant，PDA)3122或车载设备3124，终端设备可以将解码数据馈送到其显示器。对于不配备显示器的终端设备，例如STB 3116、视频会议***3118或视频监控***3120，在其中连接外部显示器3126以接收和显示解码数据。For terminal devices with their displays, for example, smartphones or tablets 3108, computers or laptops 3110, network video recorders (NVR) / digital video recorders (DVR) 3112, TV 3114, personal A digital assistant (personal digital assistant) (PDA) 3122 or a vehicle-mounted device 3124, the terminal device can feed the decoded data to its display. For terminal devices not equipped with a display, such as STB 3116, video conference system 3118, or video surveillance system 3120, an external display 3126 is connected to receive and display decoded data.
当此***中的每个设备执行编码或解码时，可以使用如上述实施例中所示的图像编码设备或图像解码设备。When each device in this system performs encoding or decoding, an image encoding device or an image decoding device as shown in the above embodiment can be used.
图12是示出终端设备3106的示例的结构的图。在终端设备3106从捕获设备3102接收到流之后，协议处理单元3202分析流的传输协议。所述协议包括但不限于实时流媒体协议(Real Time Streaming Protocol，RTSP)、超文本传送协议(Hyper Text Transfer Protocol，HTTP)、HTTP实时流媒体协议(HTTP Live streaming protocol，HLS)、MPEG-DASH、实时传输协议(Real-time Transport protocol，RTP)、实时消息传输协议(Real Time Messaging Protocol，RTMP)，或其任何种类的组合等。FIG. 12 is a diagram showing a configuration of an example of the terminal device 3106. After the terminal device 3106 receives the stream from the capture device 3102, the protocol processing unit 3202 analyzes the transmission protocol of the stream. The protocols include, but are not limited to, Real-Time Streaming Protocol (RTSP), HyperText Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG-DASH , Real-time Transport Protocol (RTP), Real-Time Messaging Protocol (RTMP), or any combination thereof.
在协议处理单元3202处理流之后，生成流文件。文件被输出到解复用单元3204。解复用单元3204可以将复用数据分离成编码音频数据和编码视频数据。如上文所述，对于其它实际场景，例如在视频会议***中，不复用编码音频数据和编码视频数据。 在这种情况下，编码数据被传输到视频解码器3206和音频解码器3208，而不通过解复用单元3204。After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 may separate the multiplexed data into encoded audio data and encoded video data. As described above, for other practical scenarios, such as in a video conference system, the encoded audio data and the encoded video data are not multiplexed. In this case, the encoded data is transmitted to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.
通过解复用处理，生成视频基本流(elementary stream，ES)、音频ES和可选的字幕。视频解码器3206，包括如上述实施例中说明的视频解码器30，通过如上述实施例中所示的解码方法对视频ES进行解码以生成视频帧，并将此数据馈送到同步单元3212。音频解码器3208对音频ES进行解码以生成音频帧，并将此数据馈送到同步单元3212。可替换地，在将视频帧馈送到同步单元3212之前可以将视频帧存储在缓冲器(图12中未示出)中。类似地，在将音频帧馈送到同步单元3212之前可以将音频帧存储在缓冲器(图12中未示出)中。Through the demultiplexing process, a video elementary stream (ES), audio ES, and optional subtitles are generated. The video decoder 3206 includes the video decoder 30 as described in the above embodiment, and decodes the video ES to generate a video frame by using the decoding method as shown in the above embodiment, and feeds this data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate an audio frame, and feeds this data to the synchronization unit 3212. Alternatively, the video frames may be stored in a buffer (not shown in FIG. 12) before the video frames are fed to the synchronization unit 3212. Similarly, the audio frames may be stored in a buffer (not shown in FIG. 12) before the audio frames are fed to the synchronization unit 3212.
同步单元3212同步视频帧和音频帧，并将视频/音频提供给视频/音频显示器3214。例如，同步单元3212同步视频和音频信息的呈现。信息可以使用与编码音频和视觉数据的呈现有关的时间戳以及与数据流本身的传送有关的时间戳而以语法进行编码。The synchronization unit 3212 synchronizes the video frame and the audio frame, and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of video and audio information. The information may be encoded in syntax using timestamps related to the presentation of the encoded audio and visual data and timestamps related to the transmission of the data stream itself.
如果流中包括字幕，则字幕解码器3210对字幕进行解码，使字幕与视频帧和音频帧同步，并将视频/音频/字幕提供给视频/音频/字幕显示器3216。If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frame and the audio frame, and provides the video / audio / subtitle to the video / audio / subtitle display 3216.
本发明不限于上述***，并且上述实施例中的图像编码设备或图像解码设备可以结合到其它***中，例如汽车***。The present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiment can be incorporated into other systems, such as a car system.
本领域技术人员能够领会，结合本文公开描述的各种说明性逻辑框、模块和算法步骤所描述的功能可以硬件、软件、固件或其任何组合来实施。如果以软件来实施，那么各种说明性逻辑框、模块、和步骤描述的功能可作为一或多个指令或代码在计算机可读媒体上存储或传输，且由基于硬件的处理单元执行。计算机可读媒体可包含计算机可读存储媒体，其对应于有形媒体，例如数据存储媒体，或包括任何促进将计算机程序从一处传送到另一处的媒体(例如，根据通信协议)的通信媒体。以此方式，计算机可读媒体大体上可对应于(1)非暂时性的有形计算机可读存储媒体，或(2)通信媒体，例如信号或载波。数据存储媒体可为可由一或多个计算机或一个或多个处理器存取以检索用于实施本申请中描述的技术的指令、代码和/或数据结构的任何可用媒体。计算机程序产品可包含计算机可读媒体。Those skilled in the art can appreciate that the functions described in connection with the various illustrative logical blocks, modules, and algorithm steps disclosed in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by the various illustrative logical blocks, modules, and steps may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another (e.g., according to a communication protocol) . In this manner, computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used to implement the techniques described in this application. The computer program product may include a computer-readable medium.
作为实例而非限制，此类计算机可读存储媒体可包括RAM、ROM、EEPROM、CD-ROM或其它光盘存储装置、磁盘存储装置或其它磁性存储装置、快闪存储器或可用来存储指令或数据结构的形式的所要程序代码并且可由计算机存取的任何其它媒体。并且，任何连接被恰当地称作计算机可读媒体。举例来说，如果使用同轴缆线、光纤缆线、双绞线、数字订户线(DSL)或例如红外线、无线电和微波等无线技术从网站、服务器或其它远程源传输指令，那么同轴缆线、光纤缆线、双绞线、DSL或例如红外线、无线电和微波等无线技术包含在媒体的定义中。但是，应理解，所述计算机可读存储媒体和数据存储媒体并不包括连接、载波、信号或其它暂时媒体，而是实际上针对于非暂时性有形存储媒体。如本文中所使用，磁盘和光盘包含压缩光盘(CD)、激光光盘、光学光盘、数字多功能光盘(DVD)和蓝光光盘，其中磁盘通常以磁性方式再现数据，而光盘利用激光以光学方式再现数据。以上各项的组合也应包含在计算机可读媒体的范围内。By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or may be used to store instructions or data structures Any form of desired program code and any other medium accessible by a computer. Also, any connection is properly termed a computer-readable medium. For example, a coaxial cable is used to transmit instructions from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media. It should be understood, however, that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are instead directed to non-transitory tangible storage media. As used herein, magnetic disks and compact discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), and Blu-ray discs, where disks typically reproduce data magnetically, and optical discs use lasers to reproduce optically data. Combinations of the above should also be included within the scope of computer-readable media.
可通过例如一或多个数字信号处理器(DSP)、通用微处理器、专用集成电路(ASIC)、 现场可编程逻辑阵列(FPGA)或其它等效集成或离散逻辑电路等一或多个处理器来执行指令。因此，如本文中所使用的术语“处理器”可指前述结构或适合于实施本文中所描述的技术的任一其它结构中的任一者。另外，在一些方面中，本文中所描述的各种说明性逻辑框、模块、和步骤所描述的功能可以提供于经配置以用于编码和解码的专用硬件和/或软件模块内，或者并入在组合编解码器中。而且，所述技术可完全实施于一或多个电路或逻辑元件中。Can be processed by one or more, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits To execute instructions. Accordingly, the term "processor" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or Into the combined codec. Moreover, the techniques can be fully implemented in one or more circuits or logic elements.
本申请的技术可在各种各样的装置或设备中实施，包含无线手持机、集成电路(IC)或一组IC(例如，芯片组)。本申请中描述各种组件、模块或单元是为了强调用于执行所揭示的技术的装置的功能方面，但未必需要由不同硬件单元实现。实际上，如上文所描述，各种单元可结合合适的软件和/或固件组合在编码解码器硬件单元中，或者通过互操作硬件单元(包含如上文所描述的一或多个处理器)来提供。The techniques of this application may be implemented in a wide variety of devices or devices, including a wireless handset, an integrated circuit (IC), or a group of ICs (eg, a chipset). Various components, modules, or units are described in this application to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in combination with suitable software and / or firmware, or through interoperable hardware units (including one or more processors as described above) provide.
以上所述，仅为本申请示例性的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。The above description is only an exemplary specific implementation of the present application, but the scope of protection of the present application is not limited to this. Any person skilled in the art can easily think of changes or changes within the technical scope disclosed in this application. Replacement shall be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims (11)

  1. 一种历史候选列表更新方法，其特征在于，所述方法包括：A method for updating a historical candidate list, characterized in that the method includes:
    获取当前块的运动信息；Get the motion information of the current block;
    获取当前块的深度信息；Get the depth information of the current block;
    获取历史候选列表；Get historical candidate list;
    若所述当前块的深度信息小于或等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。If the depth information of the current block is less than or equal to a preset threshold, the historical candidate list is updated according to the motion information of the current block.
  2. 如权利要求1所述方法，其特征在于，所述更新后的历史候选列表用于生成下一个解码块的的融合运动信息候选列表或运动矢量预测候选列表。The method according to claim 1, wherein the updated historical candidate list is used to generate a fusion motion information candidate list or a motion vector prediction candidate list for a next decoding block.
  3. 一种历史候选列表更新方法，其特征在于，所述方法包括：A method for updating a historical candidate list, characterized in that the method includes:
    获取当前块的运动信息；Get the motion information of the current block;
    获取当前块的位置；Get the position of the current block;
    获取历史候选列表；Get historical candidate list;
    若所述当前块的预设位置位于预设像素的整数倍，则根据所述当前块的运动信息对所述历史候选列表进行更新；其中，所述当前块的预设位置包括当前块的左上顶点、上边界中点、中心点、或左边界中心点。If the preset position of the current block is an integer multiple of a preset pixel, updating the historical candidate list according to the motion information of the current block; wherein the preset position of the current block includes the upper left of the current block Vertex, upper boundary midpoint, center point, or left boundary center point.
  4. 如权利要求3所述方法，其特征在于，所述更新后的历史候选列表用于生成下一个解码块的的融合运动信息候选列表或运动矢量预测候选列表。The method according to claim 3, wherein the updated historical candidate list is used to generate a fused motion information candidate list or a motion vector prediction candidate list for a next decoding block.
  5. 一种历史候选列表更新方法，其特征在于，所述方法包括：A method for updating a historical candidate list, characterized in that the method includes:
    获取当前块的运动信息；Get the motion information of the current block;
    获取当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和；Obtaining the total area of all decoded blocks in the slice where the current block is not used to update the historical candidate list since the historical candidate list was last updated;
    获取历史候选列表；Get historical candidate list;
    若所述面积总和大于等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。If the total area is greater than or equal to a preset threshold, the historical candidate list is updated according to the motion information of the current block.
  6. 如权利要求4所述方法，其特征在于，所述方法还包括：The method of claim 4, further comprising:
    将所述面积总和清0。The area sum is cleared to zero.
  7. 如权利要求4所述方法，其特征在于，所述更新后的历史候选列表用于生成下一个解码块的的融合运动信息候选列表或运动矢量预测候选列表。The method according to claim 4, wherein the updated historical candidate list is used to generate a fusion motion information candidate list or a motion vector prediction candidate list for a next decoding block.
  8. 一种历史候选列表更新装置，其特征在于，所述装置包括：A device for updating a historical candidate list, characterized in that the device includes:
    第一获取模块，用于获取当前块的运动信息；A first acquisition module, configured to acquire motion information of a current block;
    第二获取模块，用于获取当前块的深度信息；A second acquisition module, configured to acquire depth information of the current block;
    第三获取模块，用于获取历史候选列表；A third acquisition module, configured to acquire a history candidate list;
    更新模块，用于若所述当前块的深度信息小于或等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。An update module is configured to update the historical candidate list according to the motion information of the current block if the depth information of the current block is less than or equal to a preset threshold.
  9. 一种历史候选列表更新装置，其特征在于，所述装置包括：A device for updating a historical candidate list, characterized in that the device includes:
    第一获取模块，用于获取当前块的运动信息；A first acquisition module, configured to acquire motion information of a current block;
    第二获取模块，用于获取当前块的位置；A second acquisition module, configured to acquire the position of the current block;
    第三获取模块，用于获取历史候选列表；A third acquisition module, configured to acquire a history candidate list;
    更新模块，用于若所述当前块的预设位置位于预设像素的整数倍，则根据所述当前块的运动信息对所述历史候选列表进行更新；其中，所述当前块的预设位置包括当前块的左上顶点、上边界中点、中心点、或左边界中心点。An update module is configured to update the historical candidate list according to the motion information of the current block if the preset position of the current block is an integer multiple of a preset pixel; wherein the preset position of the current block is Including the upper left vertex, the upper border midpoint, the center point, or the left border center point of the current block.
  10. 一种历史候选列表更新装置，其特征在于，所述装置包括：A device for updating a historical candidate list, characterized in that the device includes:
    第一获取模块，用于获取当前块的运动信息；A first acquisition module, configured to acquire motion information of a current block;
    第二获取模块，用于获取当前块所在片中自上一次更新历史候选列表后未用于更新历史候选列表的所有解码块的面积总和；A second obtaining module, configured to obtain the total area of all decoded blocks in the slice where the current block is not used to update the historical candidate list since the historical candidate list was last updated;
    第三获取模块，用于获取历史候选列表；A third acquisition module, configured to acquire a history candidate list;
    更新模块，用于若所述面积总和大于等于预设阈值，则根据所述当前块的运动信息对所述历史候选列表进行更新。An update module is configured to update the historical candidate list according to the motion information of the current block if the total area is greater than or equal to a preset threshold.
  11. 一种解码设备，其特征在于，所述设备包括：相互耦合的存储器和处理器，所述处理器调用存储在所述存储器中的程序代码以执行如权利要求1-7中任一项所述方法。A decoding device, characterized in that the device includes a memory and a processor coupled to each other, and the processor calls program code stored in the memory to execute the program according to any one of claims 1-7. method.
PCT/CN2019/103841 2018-08-30 2019-08-30 Method and apparatus for updating historical candidate list WO2020043206A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810999655.2A CN110876058B (en) 2018-08-30 2018-08-30 Historical candidate list updating method and device
CN201810999655.2 2018-08-30

Publications (1)

Publication Number Publication Date
WO2020043206A1 true WO2020043206A1 (en) 2020-03-05

Family

ID=69643177

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/103841 WO2020043206A1 (en) 2018-08-30 2019-08-30 Method and apparatus for updating historical candidate list

Country Status (2)

Country Link
CN (1) CN110876058B (en)
WO (1) WO2020043206A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113709458A (en) * 2020-05-22 2021-11-26 腾讯科技（深圳）有限公司 Displacement vector prediction method, device and equipment in video coding and decoding

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114071158A (en) * 2020-07-29 2022-02-18 腾讯科技（深圳）有限公司 Motion information list construction method, device and equipment in video coding and decoding

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104079944A (en) * 2014-06-30 2014-10-01 华为技术有限公司 Video coding motion vector list establishing method and system
WO2015006924A1 (en) * 2013-07-16 2015-01-22 Mediatek Singapore Pte. Ltd. An additional texture merging candidate
CN106105191A (en) * 2014-03-19 2016-11-09 株式会社Kt For the method and apparatus processing multiview video signal
WO2018145600A1 (en) * 2017-02-07 2018-08-16 Mediatek Inc. Adaptive prediction candidate positions for video coding

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106851271B (en) * 2011-03-08 2019-10-18 Jvc 建伍株式会社 Moving image encoding device and moving picture encoding method
CN107197272B (en) * 2011-08-29 2019-12-06 苗太平洋控股有限公司 Method for encoding image in merge mode
KR101423648B1 (en) * 2011-09-09 2014-07-28 주식회사 케이티 Methods of decision of candidate block on inter prediction and appratuses using the same
DK2763414T3 (en) * 2011-09-29 2020-11-30 Sharp Kk PICTURE DECODING DEVICE AND PICTURE DECODING METHOD FOR PERFORMING THE CONVERSION OF BI PREDICTION TO UNI PREDICTION
TWI505696B (en) * 2011-11-08 2015-10-21 Samsung Electronics Co Ltd Method for decoding image
CN110139108B (en) * 2011-11-11 2023-07-18 Ge视频压缩有限责任公司 Apparatus and method for encoding multi-view signals into multi-view data streams
CN104272743B (en) * 2012-05-09 2019-01-01 太阳专利托管公司 Execute method, coding and decoding methods and its device of motion-vector prediction
KR101706309B1 (en) * 2013-04-10 2017-02-14 에이치에프아이 이노베이션 인크. Method and apparatus of inter-view candidate derivation for three-dimensional video coding

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015006924A1 (en) * 2013-07-16 2015-01-22 Mediatek Singapore Pte. Ltd. An additional texture merging candidate
CN106105191A (en) * 2014-03-19 2016-11-09 株式会社Kt For the method and apparatus processing multiview video signal
CN104079944A (en) * 2014-06-30 2014-10-01 华为技术有限公司 Video coding motion vector list establishing method and system
WO2018145600A1 (en) * 2017-02-07 2018-08-16 Mediatek Inc. Adaptive prediction candidate positions for video coding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
BYTEDANCE INC: "CE4-related: History-based Motion Vector Prediction", JOINT VIDEO EXPERTS TEAM (JVET) OF ITU-T SG 16 WP 3 AND ISO/IEC JTC 1/SC 29/ WG 11, 18 July 2018 (2018-07-18) *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113709458A (en) * 2020-05-22 2021-11-26 腾讯科技（深圳）有限公司 Displacement vector prediction method, device and equipment in video coding and decoding
CN113709458B (en) * 2020-05-22 2023-08-29 腾讯科技（深圳）有限公司 Displacement vector prediction method, device and equipment in video coding and decoding

Also Published As

Publication number Publication date
CN110876058B (en) 2021-09-21
CN110876058A (en) 2020-03-10

Similar Documents

Publication Publication Date Title
KR102542000B1 (en) History-based video coding method and apparatus
US20230171399A1 (en) Intra prediction-based image coding method and apparatus using mpm list
US10542280B2 (en) Encoding optimization with illumination compensation and integer motion vector restriction
US11516490B2 (en) Method and device for inter predicting on basis of DMVR
RU2620723C2 (en) Limitation with unidirectional inter-prediction for prediction blocks in b-slices
US11722659B2 (en) Affine motion prediction-based image decoding method and apparatus using affine MVP candidate list in image coding system
JP7148612B2 (en) Video data inter prediction method, apparatus, video encoder, video decoder and program
CN112673633B (en) Encoder, decoder and corresponding methods for merging modes
CN116489345A (en) Apparatus and method for conditional decoder-side motion vector correction in video coding
JP2023508571A (en) Filtering method and apparatus for cross-component linear model prediction
WO2020048502A1 (en) Method and device for bidirectional inter frame prediction
US20230291931A1 (en) Method and device for deriving inter-view motion merging candidate
WO2020043206A1 (en) Method and apparatus for updating historical candidate list
KR20230150284A (en) Efficient video encoder architecture
RU2795934C2 (en) Method and device for bidirectional inter-prediction
RU2818520C2 (en) Bidirectional inter-prediction method and device
RU2798316C2 (en) Method and equipment for external prediction
JP7436703B2 (en) Image encoding/decoding method and apparatus for signaling DPB parameters, and computer-readable recording medium storing bitstream
WO2024148492A1 (en) Reference image list construction method, video coding method, apparatus and system, and video decoding method, apparatus and system
WO2020054190A1 (en) Conversion device, decoding device, conversion method and decoding method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19855265

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19855265

Country of ref document: EP

Kind code of ref document: A1