TW202243482A - Picture orientation and quality metrics supplemental enhancement information message for video coding - Google Patents

Picture orientation and quality metrics supplemental enhancement information message for video coding Download PDF

Info

Publication number
TW202243482A
TW202243482A TW111108835A TW111108835A TW202243482A TW 202243482 A TW202243482 A TW 202243482A TW 111108835 A TW111108835 A TW 111108835A TW 111108835 A TW111108835 A TW 111108835A TW 202243482 A TW202243482 A TW 202243482A
Authority
TW
Taiwan
Prior art keywords
transformation
picture
transform
video
syntax element
Prior art date
Application number
TW111108835A
Other languages
Chinese (zh)
Inventor
永 何
穆漢麥德 傑德 柯本
丹姆裘 瑞斯諾斯基
Original Assignee
美商高通公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/653,947 external-priority patent/US20220321918A1/en
Application filed by 美商高通公司 filed Critical 美商高通公司
Publication of TW202243482A publication Critical patent/TW202243482A/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/88—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Video encoders and video decoders are configured to supplemental enhancement information (SEI) messages. The SEI messages may include picture orientation transform type syntax elements that indicate how a picture may be rotated and/or mirrored. The SEI messages may also include quality metrics.

Description

用於視訊寫碼的圖片方位和品質度量補充增強資訊訊息Image Orientation and Quality Metrics Supplementary Enhanced Information for Video Coding

本申請要求主張於2021年4月2日提交的、編號為63/170,267的美國臨時申請的權益，以及於2021年6月24日提交的、編號為63/214,378的美國臨時申請的權益，上述申請中的每個申請的全部內容通過引用的方式併入本文中。This application claims the benefit of U.S. Provisional Application No. 63/170,267, filed April 2, 2021, and U.S. Provisional Application No. 63/214,378, filed June 24, 2021, above The entire contents of each of the applications are incorporated herein by reference.

本揭露內容涉及視訊編碼和視訊解碼。This disclosure relates to video encoding and video decoding.

數位視訊能力可以被合併到寬範圍的設備中，該寬範圍的設備包括數位電視機、數位直播系統、無線廣播系統、個人數位助理(personal digital assistants, PDA)、膝上型電腦或桌上型電腦、平板電腦、電子書閱讀器、數位相機、數位記錄設備、數位媒體播放器、視訊遊戲設備、視訊遊戲機、蜂巢式或衛星無線電話、所謂的“智慧電話”、視訊電話會議設備、視訊串流傳輸設備等。數位視訊設備實施視訊寫碼(coding)技術(諸如在通過MPEG-2、MPEG-4、ITU-T H.263、ITU-T H.264/MPEG-4(部分10，先進視訊寫碼(Advanced Video Coding, AVC))、ITU-T H.265/高效率視訊寫碼(High Efficiency Video Coding, HEVC)、ITU-T H.266/通用視訊寫碼(Versatile Video Coding, VVC)定義的標準以及這樣的標準的擴展中描述的那些技術)以及專有視訊編解碼器/格式(諸如由開放媒體聯盟開發的AOMedia視訊1(AOMedia Video 1, AV1))。通過實施這樣的視訊寫碼技術，視訊設備可以更高效地發送、接收、編碼、解碼和/或儲存數位視訊資訊。Digital video capabilities can be incorporated into a wide range of devices including digital televisions, digital broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptops or desktops Computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite wireless phones, so-called "smart phones", video teleconferencing devices, video streaming equipment, etc. Digital video devices implement video coding (coding) technology (such as in MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 (Part 10, Advanced Video Coding) Video Coding, AVC)), ITU-T H.265/High Efficiency Video Coding (HEVC), ITU-T H.266/Versatile Video Coding (VVC) defined standards and extensions of such standards) as well as proprietary video codecs/formats (such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media). By implementing such video encoding technology, video equipment can more efficiently send, receive, encode, decode and/or store digital video information.

視訊寫碼技術包括空間(圖片內(intra-picture))預測和/或時間(圖片間(inter-picture))預測以減少或去除在視訊序列中固有的冗餘。對於基於區塊的視訊寫碼，視訊切片(例如，視訊圖片或視訊圖片的一部分)可以被分割為視訊區塊，該視訊區塊還可以被稱為寫碼樹單元(coding tree units, CTU)、寫碼單元(coding units, CU)和/或寫碼節點。在圖片的經幀內寫碼(intra-coded, I)的切片中的視訊區塊是使用相對於在相同圖片中的相鄰區塊中的參考樣本的空間預測來編碼的。在圖片的經幀間寫碼(P或B)的切片中的視訊區塊可以使用相對於在相同圖片中的相鄰區塊中的參考樣本的空間預測或者相對於在其它參考圖片中的參考樣本的時間預測。圖片可以被稱為幀，以及參考圖片可以被稱為參考幀。Video coding techniques include spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (eg, a video picture or a portion of a video picture) may be divided into video blocks, which may also be referred to as coding tree units (CTUs) , coding units (coding units, CU) and/or coding nodes. Video blocks in an intra-coded (I) slice of a picture are coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an intercoded (P or B) slice of a picture can use spatial prediction relative to reference samples in neighboring blocks in the same picture or relative to reference samples in other reference pictures Sample time predictions. A picture may be called a frame, and a reference picture may be called a reference frame.

概括而言，本揭露內容描述了用於對視訊資料進行寫碼的技術。具體而言，本揭露內容描述了用於對訊息(例如，補充增強資訊(supplemental enhancement information, SEI)訊息和/或其它封包化結構)進行編碼和解碼的技術，該訊息包括輔助處理(例如，解碼、顯示等)視訊資料的元資料。本揭露內容的訊息可以包括語法元素，該語法元素指示圖片的方位和/或要應用於經解碼的圖片的變換，該變換可以用於將經解碼的圖片旋轉和/或鏡像到期望的方位。語法元素可以指示用於顯示的整個圖片或組成圖片(例如，左視圖和右視圖立體圖片)的變換。在另一示例中，該訊息可以包括指示圖片品質度量的語法元素。圖片品質度量可以指示圖片的編碼品質，諸如基於品質的視埠切換和基於品質的度量測量。In summary, this disclosure describes techniques for encoding video data. Specifically, this disclosure describes techniques for encoding and decoding information (e.g., supplemental enhancement information (SEI) messages and/or other packetized structures) that includes auxiliary processing (e.g., decode, display, etc.) metadata for video data. Messages of the present disclosure may include syntax elements that indicate the orientation of a picture and/or a transformation to be applied to the decoded picture that may be used to rotate and/or mirror the decoded picture to a desired orientation. A syntax element may indicate a transformation for display of an entire picture or constituent pictures (eg, left-view and right-view stereoscopic pictures). In another example, the message may include a syntax element indicating a picture quality metric. Picture quality metrics may indicate the coding quality of a picture, such as quality-based viewport switching and quality-based metrics.

視訊解碼器或其它設備可以對訊息進行解碼，並且根據訊息來處理視訊資料的圖片。圖片方位訊息可以用於向視訊解碼器提供關於要應用於經解碼的圖片的推薦方位變換的指令。以這種方式，對經解碼的圖片的顯示可以以更合適的方位來示出。視訊解碼器可以在對經解碼的圖片的後處理中使用品質度量，和/或可以使用品質度量來選擇更高品質的圖片以用於幀間預測。A video decoder or other device decodes the message and processes the pictures of the video data based on the message. The picture orientation information can be used to provide instructions to the video decoder as to recommended orientation transforms to be applied to the decoded pictures. In this way, the display of the decoded picture can be shown in a more suitable orientation. A video decoder may use the quality metric in post-processing of the decoded picture, and/or may use the quality metric to select a higher quality picture for inter prediction.

在一個示例中，本揭露內容描述了一種處理視訊資料的方法，該方法包括：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。In one example, the present disclosure describes a method of processing video data that includes: receiving a picture; and encoding picture orientation information that includes a transform-type syntax element indicating a source from multiple The transform to apply to the image among the transforms.

在另一示例中，本揭露內容描述了一種被配置為處理視訊資料的裝置，該裝置包括：被配置為儲存圖片的記憶體；以及在電路中實施並且與該記憶體相通訊的一個或多個處理器，該一個或多個處理器被配置為：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。In another example, the present disclosure describes an apparatus configured to process video data, the apparatus including: a memory configured to store pictures; and one or more memory devices implemented in circuitry and in communication with the memory a processor, the one or more processors configured to: receive a picture; and encode a picture orientation message that includes a transform type syntax element, wherein the transform type syntax element indicates from among a plurality of transforms to apply for image transformations.

在另一示例中，本揭露內容描述了一種被配置為處理視訊資料的裝置，該裝置包括：用於接收圖片的單元；以及用於對包括變換類型語法元素的圖片方位訊息進行寫碼的單元，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。In another example, the present disclosure describes an apparatus configured to process video data, the apparatus comprising: means for receiving a picture; and means for encoding picture orientation information including transform type syntax elements , where the transform-type syntax element indicates the transform to be applied to the picture from among a plurality of transforms.

在另一示例中，本揭露內容描述了一種儲存指令的非暫時性電腦可讀儲存媒體，該指令當被執行時使得被配置為處理視訊資料的設備的一個或多個處理器進行以下操作：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。In another example, the present disclosure describes a non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to process video data to: receiving a picture; and encoding a picture orientation message including a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture.

在另一示例中，本揭露內容描述了一種處理視訊資料的方法，該方法包括：接收圖片；以及對包括品質度量語法元素的品質度量訊息進行寫碼，其中，該品質度量語法元素指示與圖片相關的品質度量的值。In another example, the present disclosure describes a method of processing video data that includes: receiving a picture; and encoding a quality metric message that includes a quality metric syntax element, wherein the quality metric syntax element indicates a The value of the associated quality metric.

在另一示例中，本揭露內容描述了一種被配置為處理視訊資料的裝置，該裝置包括：被配置為儲存圖片的記憶體；以及在電路中實施並且與該記憶體相通訊的一個或多個處理器，該一個或多個處理器被配置為：接收圖片；以及對包括品質度量語法元素的品質度量訊息進行寫碼，其中，該品質度量語法元素指示與圖片相關的品質度量的值。In another example, the present disclosure describes an apparatus configured to process video data, the apparatus including: a memory configured to store pictures; and one or more memory devices implemented in circuitry and in communication with the memory and one or more processors configured to: receive a picture; and encode a quality metric message including a quality metric syntax element indicating a value of a quality metric associated with the picture.

在另一示例中，本揭露內容描述了一種被配置為處理視訊資料的裝置，該裝置包括：用於接收圖片的單元；以及用於對包括品質度量語法元素的品質度量訊息進行寫碼的單元，其中，該品質度量語法元素指示與圖片相關的品質度量的值。In another example, the present disclosure describes an apparatus configured to process video data, the apparatus comprising: means for receiving a picture; and means for encoding a quality metric message including a quality metric syntax element , where the quality metric syntax element indicates the value of the quality metric associated with the picture.

在另一示例中，本揭露內容描述了一種儲存指令的非暫時性電腦可讀儲存媒體，該指令當被執行時使得被配置為處理視訊資料的設備的一個或多個處理器進行以下操作：接收圖片；以及對包括品質度量語法元素的品質度量訊息進行寫碼，其中，該品質度量語法元素指示與圖片相關的品質度量的值。In another example, the present disclosure describes a non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to process video data to: receiving a picture; and encoding a quality metric message including a quality metric syntax element indicating a value of a quality metric associated with the picture.

在圖式和下文的描述中闡述了一個或多個示例的細節。根據說明書、圖式和請求項，其它特徵、對象和優勢將是顯而易見的。The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects and advantages will be apparent from the description, drawings and claims.

本揭露內容描述了用於對訊息(例如，補充增強資訊(SEI)訊息和/或其它封包化結構)進行編碼和解碼的技術，該訊息包括輔助處理(例如，解碼、顯示等)視訊資料的元資料。本揭露內容的訊息可以包括語法元素，該語法元素指示圖片的方位和/或要應用於經解碼的圖片的變換，該變換可以用於將經解碼的圖片旋轉和/或鏡像到期望的方位。語法元素可以指示用於顯示的整個圖片或組成圖片(例如，左視圖和右視圖立體圖片)的變換。在另一示例中，該訊息可以包括指示圖片品質度量的語法元素。圖片品質度量可以指示圖片的編碼品質，諸如基於品質的視埠切換和基於品質的度量測量。This disclosure describes techniques for encoding and decoding information (e.g., Supplemental Enhancement Information (SEI) messages and/or other packetized structures) that includes information to assist in processing (e.g., decoding, displaying, etc.) video data metadata. Messages of the present disclosure may include syntax elements that indicate the orientation of a picture and/or a transformation to be applied to the decoded picture that may be used to rotate and/or mirror the decoded picture to a desired orientation. A syntax element may indicate a transformation for display of an entire picture or constituent pictures (eg, left-view and right-view stereoscopic pictures). In another example, the message may include a syntax element indicating a picture quality metric. Picture quality metrics may indicate the coding quality of a picture, such as quality-based viewport switching and quality-based metrics.

視訊解碼器或其它設備可以對訊息進行解碼，並且根據訊息來處理視訊資料的圖片。圖片方位訊息可以用於向視訊解碼器提供關於要應用於經解碼的圖片的推薦方位變換的指令。以這種方式，經解碼的圖片的顯示可以是以更合適的方位示出的。視訊解碼器可以在對經解碼的圖片的後處理中使用品質度量，和/或可以使用品質度量來選擇更高品質的圖片以用於幀間預測。A video decoder or other device decodes the message and processes the pictures of the video data based on the message. The picture orientation information can be used to provide instructions to the video decoder as to recommended orientation transforms to be applied to the decoded pictures. In this way, the display of the decoded picture can be shown in a more suitable orientation. A video decoder may use the quality metric in post-processing of the decoded picture, and/or may use the quality metric to select a higher quality picture for inter prediction.

圖1是示出可以執行本揭露內容的技術的示例視訊編碼和解碼系統100的方塊圖。概括而言，本揭露內容的技術針對對視訊資料進行寫碼(編碼和/或解碼)。通常，視訊資料包括用於處理視訊的任何資料。因此，視訊資料可以包括原始的未經編碼的視訊、經編碼的視訊、經解碼的(例如，經重構的)視訊、以及視訊元資料(諸如發信資料)。1 is a block diagram illustrating an example video encoding and decoding system 100 that may implement techniques of this disclosure. In general terms, the techniques of this disclosure are directed to coding (encoding and/or decoding) video data. In general, video data includes any data used to process video. Thus, video data may include raw unencoded video, encoded video, decoded (eg, reconstructed) video, and video metadata (such as signaling data).

如圖1所示，在該示例中，系統100包括源設備102，源設備102提供要被目的地設備116解碼和顯示的經編碼的視訊資料。具體地，源設備102經由電腦可讀媒體110來將視訊資料提供給目的地設備116。源設備102和目的地設備116可以包括多種多樣的設備中的任何設備，其包括桌上型電腦、筆記本電腦(即，膝上型電腦)、行動設備、平板電腦、機上盒、電話手機(諸如智慧電話)、電視機、相機、顯示設備、數位媒體播放器、視訊遊戲機、視訊串流傳輸設備、廣播接收機設備等。在一些情況下，源設備102和目的地設備116可以被配備用於無線通訊，並且因此可以被稱為無線通訊設備。As shown in FIG. 1 , in this example, system 100 includes source device 102 that provides encoded video material to be decoded and displayed by destination device 116 . Specifically, the source device 102 provides the video data to the destination device 116 via the computer-readable medium 110 . Source device 102 and destination device 116 may comprise any of a wide variety of devices including desktop computers, notebook computers (i.e., laptops), mobile devices, tablets, set-top boxes, telephone handsets ( Such as smart phones), televisions, cameras, display devices, digital media players, video game consoles, video streaming equipment, broadcast receiver equipment, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication, and thus may be referred to as wireless communication devices.

在圖1的示例中，源設備102包括視訊源104、記憶體106、視訊編碼器200和輸出介面108。目的地設備116包括輸入介面122、視訊解碼器300、記憶體120和顯示設備118。根據本揭露內容，源設備102的視訊編碼器200和目的地設備116的視訊解碼器300可以被配置為應用用於SEI訊息寫碼的技術。因此，源設備102表示視訊編碼設備的示例，而目的地設備116表示視訊解碼設備的示例。在其它示例中，源設備和目的地設備可以包括其它組件或排列。例如，源設備102可以從諸如外部相機的外部視訊源接收視訊資料。同樣，目的地設備116可以與外部顯示設備對接，而不是包括整合的顯示設備。In the example of FIG. 1 , the source device 102 includes a video source 104 , a memory 106 , a video encoder 200 and an output interface 108 . The destination device 116 includes an input interface 122 , a video decoder 300 , a memory 120 and a display device 118 . According to the present disclosure, the video encoder 200 of the source device 102 and the video decoder 300 of the destination device 116 may be configured to apply techniques for encoding SEI messages. Thus, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source and destination devices may include other components or arrangements. For example, source device 102 may receive video data from an external video source, such as an external camera. Likewise, destination device 116 may interface with an external display device rather than include an integrated display device.

如在圖1中所示的系統100僅是一個示例。通常，任何數位視訊編碼和/或解碼設備可以執行用於SEI訊息寫碼的技術。源設備102和目的地設備116僅是這樣的寫碼設備的示例，其中，源設備102生成經寫碼的視訊資料以用於傳輸給目的地設備116。本揭露內容將“寫碼”設備指代為執行對資料的寫碼(例如，編碼和/或解碼)的設備。因此，視訊編碼器200和視訊解碼器300分別表示寫碼設備(具體地，視訊編碼器和視訊解碼器)的示例。在一些示例中，源設備102和目的地設備116可以以基本上對稱的方式進行操作，使得源設備102和目的地設備116中的每一者包括視訊編碼和解碼組件。因此，系統100可以支持在源設備102和目的地設備116之間的單向或雙向視訊傳輸，例如，以用於視訊串流傳輸、視訊回放、視訊廣播或視訊電話。System 100 as shown in FIG. 1 is just one example. In general, any digital video encoding and/or decoding device can implement the techniques for encoding SEI messages. Source device 102 and destination device 116 are merely examples of encoding devices in which source device 102 generates encoded video material for transmission to destination device 116 . This disclosure refers to a "coding" device as a device that performs coding (eg, encoding and/or decoding) of material. Therefore, the video encoder 200 and the video decoder 300 respectively represent examples of encoding devices (specifically, a video encoder and a video decoder). In some examples, source device 102 and destination device 116 may operate in a substantially symmetrical manner such that source device 102 and destination device 116 each include video encoding and decoding components. Accordingly, system 100 may support one-way or two-way video transmission between source device 102 and destination device 116, eg, for video streaming, video playback, video broadcasting, or video telephony.

通常，視訊源104表示視訊資料(即，原始的未經編碼的視訊資料)的源，以及將視訊資料的一系列連續的圖片(還被稱為“幀”)提供給視訊編碼器200，視訊編碼器200對用於圖片的資料進行編碼。源設備102的視訊源104可以包括視訊捕獲設備，諸如攝像機、包含先前捕獲的原始視訊的視訊存檔、和/或用於從視訊內容提供者接收視訊的視訊饋送介面。作為另外的替代方式，視訊源104可以生成基於電腦圖形的資料作為源視訊，或者生成即時視訊、被存檔的視訊和電腦生成的視訊的組合。在每種情況下，視訊編碼器200可以對被捕獲的、預捕獲的或電腦生成的視訊資料進行編碼。視訊編碼器200可以將圖片從所接收的次序(有時被稱為“顯示次序”)重新排列為用於寫碼的寫碼次序。視訊編碼器200可以生成包括經編碼的視訊資料的位元流。然後，源設備102可以經由輸出介面108將經編碼的視訊資料輸出到電腦可讀媒體110上，以用於由例如目的地設備116的輸入介面122進行的接收和/或取回。In general, video source 104 represents a source of video data (i.e., raw unencoded video data) and provides video encoder 200 with a series of consecutive pictures (also referred to as "frames") of the video data. The encoder 200 encodes data for a picture. Video source 104 of source device 102 may include a video capture device, such as a video camera, a video archive containing previously captured raw video, and/or a video feed interface for receiving video from a video content provider. As a further alternative, video source 104 may generate computer graphics-based material as the source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 may encode captured, pre-captured, or computer-generated video material. Video encoder 200 may rearrange pictures from a received order (sometimes referred to as "display order") into a coding order for coding. The video encoder 200 can generate a bitstream including encoded video data. Source device 102 may then output the encoded video data via output interface 108 onto computer-readable medium 110 for reception and/or retrieval by, for example, input interface 122 of destination device 116 .

源設備102的記憶體106和目的地設備116的記憶體120表示通用記憶體。在一些示例中，記憶體106、120可以儲存原始的視訊資料，例如，來自視訊源104的原始視訊以及來自視訊解碼器300的原始的經解碼的視訊資料。另外或替代地，記憶體106、120可以儲存分別由例如視訊編碼器200和視訊解碼器300可執行的軟體指令。儘管記憶體106和記憶體120在該示例中是與視訊編碼器200和視訊解碼器300分開地示出的，但是應當理解的是，視訊編碼器200和視訊解碼器300還可以包括用於功能上類似或等效目的的內部記憶體。此外，記憶體106、120可以儲存例如從視訊編碼器200輸出並且輸入到視訊解碼器300的經編碼的視訊資料。在一些示例中，記憶體106、120的部分可以被分配為一個或多個視訊緩衝器，例如，以儲存原始的經解碼的和/或經編碼的視訊資料。Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory. In some examples, the memories 106 , 120 may store raw video data, eg, raw video from the video source 104 and raw decoded video data from the video decoder 300 . Additionally or alternatively, the memories 106, 120 may store software instructions executable by, for example, the video encoder 200 and the video decoder 300, respectively. Although memory 106 and memory 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include functions for internal memory for a similar or equivalent purpose. In addition, the memories 106 , 120 may store encoded video data output from the video encoder 200 and input to the video decoder 300 , for example. In some examples, portions of memory 106, 120 may be allocated as one or more video buffers, eg, to store raw decoded and/or encoded video data.

電腦可讀媒體110可以表示能夠將經編碼的視訊資料從源設備102傳送給目的地設備116的任何類型的媒體或設備。在一個示例中，電腦可讀媒體110表示通訊媒體，其使得源設備102能夠例如經由射頻網路或基於電腦的網路，來即時地向目的地設備116直接地發送經編碼的視訊資料。根據諸如無線通訊協定的通訊標準，輸出介面108可以對包括經編碼的視訊資料的傳輸訊號進行調變，以及輸入介面122可以對所接收的傳輸訊號進行解調變。通訊媒體可以包括任何無線或有線通訊媒體，諸如射頻(radio frequency, RF)頻譜或一個或多個實體傳輸線。通訊媒體可以形成諸如以下各項的基於封包的網路的一部分：區域網、廣域網、或諸如網際網路的全球網路。通訊媒體可以包括路由器、交換機、基站、或可以有助於促進從源設備102到目的地設備116的通訊的任何其它設備。Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded video material from source device 102 to destination device 116 . In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to transmit encoded video data directly to destination device 116 in real time, eg, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 can modulate the transmission signal including encoded video data, and the input interface 122 can demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. Communication media may include routers, switches, base stations, or any other device that can help facilitate communication from source device 102 to destination device 116 .

在一些示例中，源設備102可以將經編碼的資料從輸出介面108輸出到儲存設備112。類似地，目的地設備116可以經由輸入介面122從儲存設備112存取經編碼的資料。儲存設備112可以包括各種各樣的分布式或本地存取的資料儲存媒體中的任何一者，諸如硬碟驅動器、藍光光碟、DVD、CD-ROM、快閃記憶體、揮發性或非揮發性記憶體、或用於儲存經編碼的視訊資料的任何其它適當的數位儲存媒體。In some examples, source device 102 may output encoded data from output interface 108 to storage device 112 . Similarly, destination device 116 may access encoded data from storage device 112 via input interface 122 . Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray Discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

在一些示例中，源設備102可以將經編碼的視訊資料輸出到檔案伺服器114或者可以儲存由源設備102生成的經編碼的視訊資料的另一中間儲存設備。目的地設備116可以經由串流傳輸或下載來從檔案伺服器114存取被儲存的視訊資料。In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102 . The destination device 116 can access the stored video data from the file server 114 via streaming or downloading.

檔案伺服器114可以是能夠儲存經編碼的視訊資料並且將該經編碼的視訊資料發送給目的地設備116的任何類型的伺服器設備。檔案伺服器114可以表示網頁伺服器(例如，用於網站)、被配置為提供檔案傳輸協定服務(諸如檔案傳輸協定(File Transfer Protocol, FTP)或基於單向傳輸的檔案遞送(File Delivery over Unidirectional Transport, FLUTE)協定)的伺服器、內容遞送網路(content delivery network, CDN)設備、超文件傳輸協定(hypertext transfer protocol, HTTP)伺服器、多媒體廣播多播服務(Multimedia Broadcast Multicast Service, MBMS)或增強型MBMS(Enhanced MBMS, eMBMS)伺服器、和/或網路附加儲存(network attached storage, NAS)設備。檔案伺服器114可以另外或替代地實施一個或多個HTTP串流傳輸協定，諸如基於HTTP的動態自適應串流傳輸(Dynamic Adaptive Streaming over HTTP, DASH)、HTTP即時串流傳輸(HTTP Live Streaming, HLS)、即時串流傳輸協定(Real Time Streaming Protocol, RTSP)、HTTP動態串流傳輸等。File server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to destination device 116 . File server 114 may represent a web server (e.g., for a website) configured to provide file transfer protocol services (such as File Transfer Protocol (FTP) or File Delivery over Unidirectional Transport, FLUTE) agreement) server, content delivery network (content delivery network, CDN) equipment, hypertext transfer protocol (hypertext transfer protocol, HTTP) server, multimedia broadcast multicast service (Multimedia Broadcast Multicast Service, MBMS) Or an enhanced MBMS (Enhanced MBMS, eMBMS) server, and/or a network attached storage (network attached storage, NAS) device. The file server 114 may additionally or alternatively implement one or more HTTP streaming protocols, such as HTTP-based dynamic adaptive streaming (Dynamic Adaptive Streaming over HTTP, DASH), HTTP instant streaming (HTTP Live Streaming, HLS), Real Time Streaming Protocol (Real Time Streaming Protocol, RTSP), HTTP dynamic streaming, etc.

目的地設備116可以通過任何標準資料連接(包括網際網路連接)來從檔案伺服器114存取經編碼的視訊資料。這可以包括適於存取在檔案伺服器114上儲存的經編碼的視訊資料的無線通道(例如，Wi-Fi連接)、有線連接(例如，數位訂戶線(digital subscriber line, DSL)、電纜數據機等)、或這兩者的組合。輸入介面122可以被配置為根據以下各項中的任何一項或多項進行操作：上文討論的用於從檔案伺服器114取回或接收媒體資料的各種協定、或者用於取回媒體資料的其它此類協定。Destination device 116 may access the encoded video data from file server 114 through any standard data connection, including an Internet connection. This may include wireless channels (e.g., a Wi-Fi connection), wired connections (e.g., digital subscriber line (DSL), cable data) suitable for accessing encoded video data stored on the file server 114. machine, etc.), or a combination of both. Input interface 122 may be configured to operate in accordance with any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114, or the protocol for retrieving media data. other such agreements.

輸出介面108和輸入介面122可以表示無線發射機/接收機、數據機、有線聯網組件(例如，乙太網卡)、根據各種IEEE 802.11標準中的任何一種標準進行操作的無線通訊組件、或其它實體組件。在輸出介面108和輸入介面122包括無線組件的示例中，輸出介面108和輸入介面122可以被配置為根據蜂巢式通訊標準(諸如4G、4G-LTE(長期演進)、改進的LTE、5G等)來傳輸資料(諸如經編碼的視訊資料)。在輸出介面108包括無線發射機的一些示例中，輸出介面108和輸入介面122可以被配置為根據其它無線標準(諸如IEEE 802.11規範、IEEE 802.15規範(例如，ZigBee™)、藍牙™標準等)來傳輸資料(諸如經編碼的視訊資料)。在一些示例中，源設備102和/或目的地設備116可以包括相應的單晶片系統(system-on-a-chip, SoC)設備。例如，源設備102可以包括用於執行歸因於視訊編碼器200和/或輸出介面108的功能的SoC設備，並且目的地設備116可以包括用於執行歸因於視訊解碼器300和/或輸入介面122的功能的SoC設備。Output interface 108 and input interface 122 may represent wireless transmitter/receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components operating in accordance with any of the various IEEE 802.11 standards, or other entities components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to conform to a cellular communication standard (such as 4G, 4G-LTE (Long Term Evolution), LTE-Advanced, 5G, etc.) to transmit data (such as encoded video data). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to communicate in accordance with other wireless standards (such as IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee™), Bluetooth™ standards, etc.). Transmission of data (such as encoded video data). In some examples, source device 102 and/or destination device 116 may include corresponding system-on-a-chip (SoC) devices. For example, source device 102 may include a SoC device for performing functions attributed to video encoder 200 and/or output interface 108, and destination device 116 may include a device for performing functions attributed to video decoder 300 and/or input interface 108. interface 122 functions of the SoC device.

本揭露內容的技術可以應用於視訊寫碼，以支持各種多媒體應用中的任何一者，諸如空中電視廣播、有線電視傳輸、衛星電視傳輸、網際網路串流視訊傳輸(諸如基於HTTP的動態自適應串流傳輸(DASH))、被編碼到資料儲存媒體上的數位視訊、對在資料儲存媒體上儲存的數位視訊的解碼、或其它應用。The techniques of the present disclosure can be applied to video coding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as HTTP-based dynamic auto Adaptive Streaming (DASH)), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

目的地設備116的輸入介面122從電腦可讀媒體110(例如，通訊媒體、儲存設備112、檔案伺服器114等)接收經編碼的視訊位元流。經編碼的視訊位元流可以包括由視訊編碼器200定義的諸如以下語法元素的發信資訊(其還被視訊解碼器300使用)：該語法元素具有描述視訊區塊或其它寫碼單元(例如，切片、圖片、圖片組、序列等)的特性和/或處理的值。顯示設備118將經解碼的視訊資料的經解碼的圖片顯示給用戶。顯示設備118可以表示各種顯示設備中的任何一者，諸如液晶顯示器(liquid crystal display, LCD)、電漿顯示器、有機發光二極體(organic light emitting diode, OLED)顯示器、或另一類型的顯示設備。The input interface 122 of the destination device 116 receives an encoded video bitstream from the computer-readable medium 110 (eg, communication medium, storage device 112 , file server 114 , etc.). The encoded video bitstream may include signaling information defined by the video encoder 200 (which is also used by the video decoder 300) such as syntax elements that describe video blocks or other coding units (e.g. , slices, pictures, picturegroups, sequences, etc.) properties and/or processing values. The display device 118 displays the decoded pictures of the decoded video data to the user. Display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display equipment.

儘管在圖1中未示出，但是在一些示例中，視訊編碼器200和視訊解碼器300可以各自與音訊編碼器和/或音訊解碼器整合，並且可以包括適當的MUX-DEMUX單元或其它硬體和/或軟體，以處理包括公共資料串流中的音訊和視訊兩者的經多工的串流。Although not shown in FIG. 1 , in some examples video encoder 200 and video decoder 300 may each be integrated with an audio encoder and/or audio decoder and may include appropriate MUX-DEMUX units or other hardware and/or software to process a multiplexed stream including both audio and video in the common data stream.

視訊編碼器200和視訊解碼器300各自可以被實施為各種適當的編碼器和/或解碼器電路中的任何一者，諸如一個或多個微處理器、數位訊號處理器(digital signal processors, DSP)、特殊應用積體電路(application specific integrated circuits, ASIC)、現場可程式化閘陣列(field programmable gate arrays, FPGA)、離散邏輯、軟體、硬體、韌體、或其任何組合。當該技術用軟體部分地實施時，設備可以將用於軟體的指令儲存在適當的非暫時性電腦可讀媒體中，並且使用一個或多個處理器在硬體中執行指令以執行本揭露內容的技術。視訊編碼器200和視訊解碼器300中的每一者可以被包括在一個或多個編碼器或解碼器中，該一個或多個編碼器或解碼器中的任一者可以被整合為相應設備中的組合編碼器/解碼器(combined encoder/decoder, CODEC)的一部分。包括視訊編碼器200和/或視訊解碼器300的設備可以包括積體電路、微處理器、和/或無線通訊設備(諸如蜂巢式電話)。Each of video encoder 200 and video decoder 300 may be implemented as any one of various suitable encoder and/or decoder circuits, such as one or more microprocessors, digital signal processors (digital signal processors, DSPs) ), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is implemented in part in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the present disclosure. Technology. Each of the video encoder 200 and the video decoder 300 may be included in one or more encoders or decoders, any of which may be integrated into a corresponding device Part of the combined encoder/decoder (CODEC) in . Devices including video encoder 200 and/or video decoder 300 may include integrated circuits, microprocessors, and/or wireless communication devices (such as cellular phones).

視訊編碼器200和視訊解碼器300可以根據視訊寫碼標準(諸如ITU-T H.265(還被稱為高效率視訊寫碼(HEVC))或對其的擴展(諸如多視圖和/或可縮放視訊寫碼擴展))進行操作。替代地，視訊編碼器200和視訊解碼器300可以根據其它專有或行業標準(諸如ITU-T H.266標準，還被稱為通用視訊寫碼(VVC))進行操作。在其它示例中，視訊編碼器200和視訊解碼器300可以根據專有視訊編解碼器/格式(諸如AOMedia video 1(AV1)、AV1的擴展和/或AV1的後續版本(例如，AV2))進行操作。在其它示例中，視訊編碼器200和視訊解碼器300可以根據其它專有格式或行業標準來操作。然而，本揭露內容的技術不限於任何特定的寫碼標準或格式。通常，視訊編碼器200和視訊解碼器300可以被配置為結合使用SEI訊息來確定圖片方位和/或圖片品質度量的任何視訊寫碼技術來執行本揭露內容的技術。Video encoder 200 and video decoder 300 may be based on video coding standards such as ITU-T H.265 (also known as High Efficiency Video Coding (HEVC)) or extensions thereof such as multi-view and/or scalable Zoom video coding extension)) to operate. Alternatively, video encoder 200 and video decoder 300 may operate according to other proprietary or industry standards, such as the ITU-T H.266 standard, also known as Universal Video Coding (VVC). In other examples, video encoder 200 and video decoder 300 may operate in accordance with a proprietary video codec/format such as AOMedia video 1 (AV1), an extension of AV1, and/or a successor version of AV1 (e.g., AV2) operate. In other examples, video encoder 200 and video decoder 300 may operate according to other proprietary formats or industry standards. However, the techniques of this disclosure are not limited to any particular coding standard or format. In general, video encoder 200 and video decoder 300 may be configured to perform the techniques of this disclosure in conjunction with any video coding technique that uses SEI information to determine picture orientation and/or picture quality metrics.

通常，視訊編碼器200和視訊解碼器300可以執行對圖片的基於區塊的寫碼。術語“區塊”通常指代包括要被處理的(例如，在編碼和/或解碼過程中的被編碼的、被解碼的或以其它方式被使用的)資料的結構。例如，區塊可以包括亮度和/或色度資料的樣本的二維矩陣。通常，視訊編碼器200和視訊解碼器300可以對以YUV(例如，Y、Cb、Cr)格式表示的視訊資料進行寫碼。也就是說，視訊編碼器200和視訊解碼器300可以對亮度分量和色度分量進行寫碼，而不是對用於圖片的樣本的紅色、綠色和藍色(red, green, and blue, RGB)資料進行寫碼，其中，色度分量可以包括紅色色相色度分量和藍色色相色度分量兩者。在一些示例中，視訊編碼器200在進行編碼之前將所接收的經RGB格式化的資料轉換為YUV表示，並且視訊解碼器300將YUV表示轉換為RGB格式。替代地，預處理和後處理單元(未示出)可以執行這些轉換。In general, video encoder 200 and video decoder 300 may perform block-based coding of pictures. The term "block" generally refers to a structure that includes material to be processed (eg, encoded, decoded, or otherwise used in an encoding and/or decoding process). For example, a tile may comprise a two-dimensional matrix of samples of luma and/or chrominance data. Generally, the video encoder 200 and the video decoder 300 can encode video data expressed in YUV (eg, Y, Cb, Cr) format. That is, video encoder 200 and video decoder 300 may encode luma and chrominance components instead of red, green, and blue (RGB) for samples of a picture. data, wherein the chroma components may include both red hue chroma components and blue hue chroma components. In some examples, video encoder 200 converts received RGB formatted data to a YUV representation prior to encoding, and video decoder 300 converts the YUV representation to RGB format. Alternatively, preprocessing and postprocessing units (not shown) may perform these transformations.

概括而言，本揭露內容可以涉及對圖片的寫碼(例如，編碼和解碼)以包括對圖片的資料進行編碼或解碼的過程。類似地，本揭露內容可以涉及對圖片的區塊的寫碼以包括對用於區塊的資料進行編碼或解碼的過程(例如，預測和/或殘差寫碼)。經編碼的視訊位元流通常包括用於表示寫碼決策(例如，寫碼模式)以及將圖片分割為區塊的語法元素的一系列值。因此，對圖片或區塊進行寫碼的引用通常應當被理解為對用於形成圖片或區塊的語法元素的值進行寫碼。In general terms, the present disclosure may relate to the encoding (eg, encoding and decoding) of pictures to include the process of encoding or decoding data of the pictures. Similarly, the present disclosure may relate to the coding of blocks of pictures to include the process of encoding or decoding data for the blocks (eg, predictive and/or residual coding). A coded video bitstream typically includes a series of values for syntax elements representing coding decisions (eg, coding mode) and partitioning a picture into blocks. Therefore, a reference to encode a picture or block should generally be understood as encoding the value of a syntax element used to form the picture or block.

HEVC定義了各種區塊，其包括寫碼單元(CU)、預測單元(prediction units, PU)和變換單元(transform units, TU)。根據HEVC，視訊寫碼器(諸如視訊編碼器200)根據四元樹結構來將寫碼樹單元(CTU)分割為CU。也就是說，視訊寫碼器將CTU和CU分割為四個相等的、不重疊的正方形，並且四元樹的每個節點具有零個或者四個子節點。沒有子節點的節點可以被稱為“葉節點”，並且這樣的葉節點的CU可以包括一個或多個PU和/或一個或多個TU。視訊寫碼器可以進一步分割PU和TU。例如，在HEVC中，殘差四元樹(residual quadtree, RQT)表示對TU的分割。在HEVC中，PU表示幀間預測資料，而TU表示殘差資料。經幀內預測的CU包括幀內預測資訊，諸如幀內模式指示。HEVC defines various blocks, which include coding units (CU), prediction units (prediction units, PU) and transformation units (transform units, TU). According to HEVC, a video coder such as video encoder 200 partitions a coding tree unit (CTU) into CUs according to a quad tree structure. That is, the video encoder partitions the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has either zero or four child nodes. A node with no child nodes may be referred to as a "leaf node," and a CU of such a leaf node may include one or more PUs and/or one or more TUs. The video codec can further split PU and TU. For example, in HEVC, a residual quadtree (RQT) represents the division of a TU. In HEVC, PU represents inter prediction data, and TU represents residual data. An intra-predicted CU includes intra-prediction information, such as an intra-mode indication.

作為另一示例，視訊編碼器200和視訊解碼器300可以被配置為根據VVC進行操作。根據VVC，視訊寫碼器(諸如視訊編碼器200)將圖片分割為多個寫碼樹單元(CTU)。視訊編碼器200可以根據樹結構(諸如四元樹-二元樹(quadtree-binary tree, QTBT)結構或多類型樹(Multi-Type Tree, MTT)結構)來分割CTU。QTBT結構去除了多種分割類型的概念，諸如在HEVC的CU、PU和TU之間的分隔。QTBT結構包括兩個級別：根據四元樹分割而分割的第一級別、以及根據二元樹分割而分割的第二級別。QTBT結構的根節點對應於CTU。二元樹的葉節點對應於寫碼單元(CU)。As another example, video encoder 200 and video decoder 300 may be configured to operate according to VVC. According to VVC, a video encoder, such as video encoder 200, divides a picture into a plurality of coding tree units (CTUs). The video encoder 200 may partition the CTU according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (Multi-Type Tree, MTT) structure). The QTBT structure removes the concept of multiple partition types, such as partitions between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level split according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

在MTT分割結構中，可以使用四元樹(quadtree, QT)分割、二元樹(binary tree, BT)分割以及一種或多種類型的三元樹(riple tree, TT)(還被稱為三元樹(TT))分割來對區塊進行分割。三元樹或三元樹分割是其中區塊被分為三個子區塊的分割。在一些示例中，三元樹或三元樹分割將區塊劃分為三個子區塊，而不通過中心劃分原始區塊。MTT中的分割類型(例如，QT、BT和TT)可以是對稱的或不對稱的。In the MTT partition structure, quadtree (quadtree, QT) partition, binary tree (binary tree, BT) partition and one or more types of triple tree (riple tree, TT) (also known as triple Tree (TT)) split to split the blocks. A ternary tree or ternary tree partition is a partition in which a block is divided into three sub-blocks. In some examples, a ternary tree or ternary tree partitioning divides a block into three sub-blocks without dividing the original block by a center. Segmentation types (eg, QT, BT, and TT) in MTT can be symmetric or asymmetric.

當根據AV1編解碼器進行操作時，視訊編碼器200和視訊解碼器300可以被配置為以區塊來對視訊資料進行寫碼。在AV1中，可以處理的最大寫碼區塊被稱為超級區塊。在AV1中，超級區塊可以是128x128個亮度樣本或者64x64 個亮度樣本。然而，在後續視訊寫碼格式(例如，AV2)中，超級區塊可以由不同的(例如，更大的)亮度樣本大小來定義。在一些示例中，超級區塊是區塊四元樹的頂部級別。視訊編碼器200可以進一步將超級區塊分割為較小的寫碼區塊。視訊編碼器200可以使用正方形或非正方形分割來將超級區塊和其它寫碼區塊分割為較小的區塊。非正方形區塊可以包括N/2xN、NxN/2、N/4xN和NxN/4區塊。視訊編碼器200和視訊解碼器300可以對寫碼區塊中的每一者執行單獨的預測和變換過程。When operating according to the AV1 codec, the video encoder 200 and the video decoder 300 may be configured to encode video data in blocks. In AV1, the largest code block that can be processed is called a super block. In AV1, a superblock can be 128x128 luma samples or 64x64 luma samples. However, in subsequent video encoding formats (eg, AV2), superblocks may be defined by different (eg, larger) luma sample sizes. In some examples, the superblock is the top level of the block quadtree. The video encoder 200 can further divide the superblock into smaller coding blocks. Video encoder 200 may use square or non-square partitioning to partition superblocks and other coding blocks into smaller blocks. Non-square blocks may include N/2xN, NxN/2, N/4xN and NxN/4 blocks. Video encoder 200 and video decoder 300 may perform separate prediction and transformation processes on each of the coded blocks.

AV1還定義了視訊資料的瓦片(tile)。瓦片是超級區塊的矩形陣列，其可以獨立於其它瓦片進行寫碼。也就是說，視訊編碼器200和視訊解碼器300可以分別對瓦片內的寫碼區塊進行編碼和解碼，而不使用來自其它瓦片的視訊資料。然而，視訊編碼器200和視訊解碼器300可以跨瓦片邊界來執行濾波。瓦片在大小方面可以是均勻的或不均勻的。基於瓦片的寫碼可以實施用於編碼器和解碼器實施的並行處理和/或多線程。AV1 also defines tiles for video data. A tile is a rectangular array of superblocks that can be coded independently of other tiles. That is, the video encoder 200 and the video decoder 300 can respectively encode and decode coded blocks within a tile without using video data from other tiles. However, video encoder 200 and video decoder 300 may perform filtering across tile boundaries. Tiles can be uniform or non-uniform in size. Tile-based coding can implement parallel processing and/or multi-threading for encoder and decoder implementations.

在一些示例中，視訊編碼器200和視訊解碼器300可以使用單個QTBT或MTT結構來表示亮度分量和色度分量中的每一者，而在其它示例中，視訊編碼器200和視訊解碼器300可以使用兩個或更多個QTBT結構或MTT結構，諸如用於亮度分量的一個QTBT/MTT結構以及用於兩個色度分量的另一個QTBT/MTT結構(或者用於相應色度分量的兩個QTBT/MTT結構)。In some examples, video encoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chrominance components, while in other examples video encoder 200 and video decoder 300 Two or more QTBT structures or MTT structures may be used, such as one QTBT/MTT structure for the luma component and another QTBT/MTT structure for the two chroma components (or two QTBT/MTT structures for the corresponding chroma components). a QTBT/MTT structure).

視訊編碼器200和視訊解碼器300可以被配置為使用四元樹分割、QTBT分割、MTT分割、超級區塊分割或其它分割結構。Video encoder 200 and video decoder 300 may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, superblock partitioning, or other partitioning structures.

在一些示例中，CTU包括具有三個樣本陣列的圖片的亮度樣本的寫碼樹區塊(CTB)、色度樣本的兩個對應CTB、或者單色圖片或使用三個單獨的顏色平面來寫碼的圖片的樣本的CTB、以及用於對樣本進行寫碼的語法結構。CTB可以是針對N的某個值的NxN樣本區塊，使得將分量劃分為CTB是一分割。分量是來自以4:2:0、4:2:2或4:4:4顏色格式來構成圖片的三個陣列(亮度和兩個色度)中的一者的陣列或單個樣本、或以單色格式構成圖片的陣列或該陣列的單個樣本。在一些示例中，寫碼區塊是針對M和N的一些值的MxN樣本區塊，使得將CTB劃分為寫碼區塊是一分割。In some examples, a CTU includes a written coding tree block (CTB) of luma samples for a picture with three sample arrays, two corresponding CTBs of chroma samples, or a monochrome picture or written using three separate color planes. The CTB of a sample of a picture of the code, and the syntax structure used to code the sample. A CTB may be an NxN block of samples for some value of N, such that dividing a component into a CTB is a partition. Components are arrays or single samples from one of the three arrays (luminance and two chrominance) that make up the picture in 4:2:0, 4:2:2, or 4:4:4 color format, or in A monochrome format constitutes an array of pictures or a single sample of the array. In some examples, the write code blocks are MxN sample blocks for some values of M and N, such that dividing the CTB into write code blocks is a partition.

可以以各種方式在圖片中對區塊(例如，CTU或CU)進行分組。作為一個示例，磚型區(brick)可以指代在圖片中的特定瓦片內的CTU列的矩形區域。瓦片可以是在圖片中的特定瓦片行和特定瓦片列內的CTU的矩形區域。瓦片行指代CTU的矩形區域，其具有與圖片的高度相等的高度以及由語法元素(例如，諸如在圖片參數集中)指定的寬度。瓦片列指代CTU的矩形區域，其具有由語法元素(例如，諸如在圖片參數集中)指定的高度以及與圖片的寬度相等的寬度。Blocks (eg, CTUs or CUs) can be grouped in a picture in various ways. As one example, a brick may refer to a rectangular area of CTU columns within a particular tile in a picture. A tile may be a rectangular area of a CTU within a specific tile row and a specific tile column in a picture. A tile row refers to a rectangular area of a CTU that has a height equal to that of a picture and a width specified by a syntax element (eg, such as in a picture parameter set). A tile column refers to a rectangular area of a CTU that has a height specified by a syntax element (eg, such as in a picture parameter set) and a width equal to the width of the picture.

在一些示例中，可以將瓦片分割為多個磚型區，該多個磚型區中的每個磚型區可以包括在瓦片內的一個或多個CTU列。沒有被分割為多個磚型區的瓦片還可以被稱為磚型區。然而，作為瓦片的真實子集的磚型區可以不被稱為瓦片。圖片中的磚型區還可以是以切片來排列的。切片可以是圖片的整數個磚型區，其可以唯一地被包含在單個網路抽象層(network abstraction layer, NAL)單元中。在一些示例中，切片包括數個完整的瓦片或者僅一個瓦片的連續序列的完整磚型區。In some examples, a tile may be partitioned into multiple bricks, each of which may include one or more CTU columns within the tile. A tile that is not divided into tiles may also be referred to as a tile. However, brick-shaped regions, which are a true subset of tiles, may not be called tiles. The brick-shaped areas in the picture can also be arranged in slices. A slice may be an integer number of bricks of a picture that may be uniquely contained within a single network abstraction layer (NAL) unit. In some examples, a slice includes a contiguous sequence of complete tiles of several complete tiles or only one tile.

本揭露內容可以可互換地使用“NxN”和“N乘N”來指代區塊(諸如CU或其它視訊區塊)在垂直維度和水平維度方面的樣本大小，例如，16x16個樣本或16乘16個樣本。通常，16x16 CU在垂直方向上將具有16個樣本(y = 16)，並且在水平方向上將具有16個樣本(x = 16)。同樣地，NxN CU通常在垂直方向上具有N個樣本，並且在水平方向上具有N個樣本，其中N表示非負整數值。CU中的樣本可以按列和行來排列。此外，CU不一定需要在水平方向上具有與在垂直方向上相同的數量的樣本。例如，CU可以包括NxM個樣本，其中M不一定等於N。This disclosure may use "NxN" and "N by N" interchangeably to refer to the sample size of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions, e.g., 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples vertically (y=16) and 16 samples horizontally (x=16). Likewise, an NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in columns and rows. Also, a CU does not necessarily need to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include NxM samples, where M is not necessarily equal to N.

視訊編碼器200對用於CU的表示預測和/或殘差資訊以及其它資訊的視訊資料進行編碼。預測資訊指示將如何預測CU以便形成用於CU的預測區塊。殘差資訊通常表示在編碼之前的CU的樣本與預測區塊之間的逐樣本差。Video encoder 200 encodes video data representing prediction and/or residual information, among other information, for a CU. The prediction information indicates how the CU will be predicted in order to form the prediction block for the CU. The residual information typically represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

為了預測CU，視訊編碼器200通常可以通過幀間預測或幀內預測來形成用於CU的預測區塊。幀間預測通常指代根據先前寫碼的圖片的資料來預測CU，而幀內預測通常指代根據相同圖片的先前寫碼的資料來預測CU。為了執行幀間預測，視訊編碼器200可以使用一個或多個運動向量來生成預測區塊。視訊編碼器200通常可以執行運動搜索，以識別例如在CU與參考區塊之間的差異方面與CU緊密匹配的參考區塊。視訊編碼器200可以使用絕對差的總和(sum of absolute difference, SAD)、平方差的總和(sum of squared differences, SSD)、平均絕對差(mean absolute difference, MAD)、均方差(mean squared differences, MSD)、或其它這樣的差計算來計算差度量，以確定參考區塊是否與當前CU緊密匹配。在一些示例中，視訊編碼器200可以使用單向預測或雙向預測來預測當前CU。In order to predict a CU, the video encoder 200 can usually form a prediction block for the CU through inter prediction or intra prediction. Inter prediction generally refers to predicting a CU based on previously coded data for a picture, while intra prediction generally refers to predicting a CU based on previously coded data for the same picture. To perform inter prediction, video encoder 200 may use one or more motion vectors to generate a predicted block. Video encoder 200 may typically perform a motion search to identify a reference block that closely matches the CU, eg, in terms of differences between the CU and the reference block. The video encoder 200 can use sum of absolute difference (sum of absolute difference, SAD), sum of squared difference (sum of squared differences, SSD), mean absolute difference (mean absolute difference, MAD), mean squared difference (mean squared differences, MSD), or other such difference calculations to calculate the difference metric to determine whether the reference block closely matches the current CU. In some examples, video encoder 200 may predict the current CU using uni-prediction or bi-prediction.

VVC的一些示例還提供仿射運動補償模式，其可以被認為是幀間預測模式。在仿射運動補償模式下，視訊編碼器200可以確定表示非平移運動(諸如放大或縮小、旋轉、透視運動或其它不規則的運動類型)的兩個或更多個運動向量。Some examples of VVC also provide an affine motion compensation mode, which can be thought of as an inter prediction mode. In the affine motion compensation mode, video encoder 200 may determine two or more motion vectors representing non-translational motion, such as zooming in or out, rotation, perspective motion, or other irregular types of motion.

為了執行幀內預測，視訊編碼器200可以選擇幀內預測模式來生成預測區塊。VVC的一些示例提供六十七個幀內預測模式，其包括各種方向性模式、以及平面模式和DC模式。通常，視訊編碼器200選擇幀內預測模式，幀內預測模式描述與當前區塊(例如，CU的區塊)相鄰的樣本，根據該相鄰的樣本來預測當前區塊的樣本。假定視訊編碼器200以光柵掃描次序(左到右、上到下)對CTU和CU進行寫碼，這樣的樣本通常可以是在與當前區塊相同的圖片中在當前區塊的上方、左上方或左側。To perform intra prediction, the video encoder 200 may select an intra prediction mode to generate a prediction block. Some examples of VVC provide sixty-seven intra prediction modes, including various directional modes, as well as planar and DC modes. Generally, the video encoder 200 selects an intra prediction mode, which describes samples adjacent to a current block (eg, a block of a CU), and samples of the current block are predicted according to the adjacent samples. Assuming video encoder 200 codes CTUs and CUs in raster scan order (left-to-right, top-to-bottom), such samples can typically be above, top left of the current block in the same picture as the current block or left.

視訊編碼器200對表示用於當前區塊的預測模式的資料進行編碼。例如，對於幀間預測模式，視訊編碼器200可以對表示使用各種可用幀間預測模式中的哪一種預測模式的資料以及用於對應模式的運動資訊進行編碼。對於單向或雙向幀間預測，例如，視訊編碼器200可以使用先進運動向量預測(advanced motion vector prediction, AMVP)或合併模式來對運動向量進行編碼。視訊編碼器200可以使用類似的模式來對用於仿射運動補償模式的運動向量進行編碼。The video encoder 200 encodes data representing the prediction mode for the current block. For example, for an inter prediction mode, video encoder 200 may encode data indicating which of various available inter prediction modes to use and motion information for the corresponding mode. For unidirectional or bidirectional inter prediction, for example, the video encoder 200 may use advanced motion vector prediction (AMVP) or merge mode to encode the motion vector. Video encoder 200 may use a similar scheme to encode motion vectors for the affine motion compensation mode.

AV1包括用於對視訊資料的寫碼區塊進行編碼和解碼的兩個通用技術。這兩個通用技術是幀內預測(例如，幀內預測或空間預測)和幀間預測(例如，幀間預測或時間預測)。在AV1的背景下，當使用幀內預測模式來預測視訊資料的當前幀的區塊時，視訊編碼器200和視訊解碼器300不使用來自視訊資料的其它幀的視訊資料。對於大多數幀內預測模式，視訊編碼器200基於在當前區塊中的樣本值與根據相同幀中的參考樣本生成的預測值之間的差來對當前幀的區塊進行編碼。視訊編碼器200確定基於幀內預測模式根據參考樣本而生成的預測值。AV1 includes two general techniques for encoding and decoding coded blocks of video data. The two general techniques are intra prediction (eg, intra prediction or spatial prediction) and inter prediction (eg, inter prediction or temporal prediction). In the context of AV1, the video encoder 200 and the video decoder 300 do not use video data from other frames of video data when using intra prediction mode to predict blocks of the current frame of video data. For most intra prediction modes, video encoder 200 encodes the block of the current frame based on the difference between the sample values in the current block and the predicted values generated from reference samples in the same frame. The video encoder 200 determines a prediction value generated from reference samples based on an intra prediction mode.

在諸如對區塊的幀內預測或幀間預測的預測之後，視訊編碼器200可以計算用於該區塊的殘差資料。殘差資料(諸如殘差區塊)表示在區塊與用於該區塊的預測區塊之間的逐樣本差，該預測區塊是使用對應的預測模式來形成的。視訊編碼器200可以將一個或多個變換應用於殘差區塊，以在變換域中而不是在樣本域中產生經變換的資料。例如，視訊編碼器200可以將離散餘弦變換(discrete cosine transform, DCT)、整數變換、小波變換或概念上類似的變換應用於殘差視訊資料。另外，視訊編碼器200可以在第一變換之後應用二次變換，諸如模式相關的不可分離二次變換(mode-dependent non-separable secondary transform, MDNSST)、訊號相關變換、Karhunen-Loeve變換(Karhunen-Loeve transform, KLT)等。視訊編碼器200在應用一個或多個變換之後產生變換係數。After prediction such as intra prediction or inter prediction for a block, video encoder 200 may calculate residual data for the block. Residual data, such as a residual block, represents the sample-by-sample difference between a block and the predicted block for that block, which was formed using the corresponding prediction mode. Video encoder 200 may apply one or more transforms to the residual block to generate transformed data in the transform domain instead of the sample domain. For example, video encoder 200 may apply discrete cosine transform (DCT), integer transform, wavelet transform, or conceptually similar transforms to the residual video data. In addition, video encoder 200 may apply a secondary transform after the first transform, such as mode-dependent non-separable secondary transform (MDNSST), signal-dependent transform, Karhunen-Loeve transform (Karhunen-Loeve transform). Loeve transform, KLT) and so on. Video encoder 200 generates transform coefficients after applying one or more transforms.

如上所述，在用於產生變換係數的任何變換之後，視訊編碼器200可以執行對變換係數的量化。量化通常指代如下的過程：在該過程中，對變換係數進行量化以可能地減少用於表示變換係數的資料量，從而提供進一步的壓縮。通過執行量化過程，視訊編碼器200可以減小與變換係數中的一些或所有變換係數相關聯的位元深度。例如，視訊編碼器200可以在量化期間將n位元值向下舍入為m位元值，其中n大於m。在一些示例中，為了執行量化，視訊編碼器200可以執行對要被量化的值的按位右移。As described above, following any transform used to generate the transform coefficients, video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to the process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing a quantization process, video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, video encoder 200 may round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, video encoder 200 may perform a bitwise right shift of the value to be quantized.

在量化之後，視訊編碼器200可以掃描變換係數，從而從包括經量化的變換係數的二維矩陣產生一維向量。掃描可以被設計為將較高能量(並且因此較低頻率)的變換係數放在向量的前面，並且將較低能量(並且因此較高頻率)的變換係數放在向量的後面。在一些示例中，視訊編碼器200可以利用預定義的掃描次序來掃描經量化的變換係數以產生經序列化的向量，並且然後對向量的經量化的變換係數進行熵編碼。在其它示例中，視訊編碼器200可以執行自適應掃描。在掃描經量化的變換係數以形成一維向量之後，視訊編碼器200可以例如根據上下文自適應二進制算術寫碼(context-adaptive binary arithmetic coding, CABAC)來對一維向量進行熵編碼。視訊編碼器200還可以對用於描述與經編碼的視訊資料相關聯的元資料的語法元素的值進行熵編碼，以供視訊解碼器300在對視訊資料進行解碼時使用。After quantization, video encoder 200 may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including quantized transform coefficients. The scan can be designed to place higher energy (and therefore lower frequency) transform coefficients at the front of the vector, and lower energy (and thus higher frequency) transform coefficients at the back of the vector. In some examples, video encoder 200 may scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 may perform entropy coding on the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 can also entropy encode values of syntax elements used to describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

為了執行CABAC，視訊編碼器200可以將在上下文模型內的上下文指派給要被發送的符號。上下文可以涉及例如符號的相鄰值是否為零值。概率確定可以是基於被指派給符號的上下文的。To perform CABAC, video encoder 200 may assign contexts within a context model to symbols to be transmitted. The context may relate, for example, to whether a neighboring value of a symbol is zero-valued. The probability determination may be based on the context assigned to the symbol.

視訊編碼器200還可以例如在圖片標頭、區塊標頭、切片標頭中生成去往視訊解碼器300的語法資料(諸如基於區塊的語法資料、基於圖片的語法資料和基於序列的語法資料)、或其它語法資料(諸如序列參數集(sequence parameter set, SPS)、圖片參數集(picture parameter set, PPS)或視訊參數集(video parameter set, VPS))。同樣地，視訊解碼器300可以對此類語法資料進行解碼以確定如何對對應的視訊資料進行解碼。Video encoder 200 may also generate syntax data (such as block-based syntax data, picture-based syntax data, and sequence-based syntax data) to video decoder 300, e.g., in picture headers, block headers, slice headers data), or other syntax data (such as sequence parameter set (sequence parameter set, SPS), picture parameter set (picture parameter set, PPS) or video parameter set (video parameter set, VPS)). Likewise, the video decoder 300 can decode such syntax data to determine how to decode the corresponding video data.

以這種方式，視訊編碼器200可以生成位元流，其包括經編碼的視訊資料，例如，描述將圖片分割為區塊(例如，CU)以及用於該區塊的預測和/或殘差資訊的語法元素。最終，視訊解碼器300可以接收位元流並且對經編碼的視訊資料進行解碼。In this manner, video encoder 200 may generate a bitstream that includes encoded video data, e.g., describing partitioning of a picture into blocks (e.g., CUs) and prediction and/or residuals for the blocks. Syntactic elements for information. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

通常，視訊解碼器300執行與由視訊編碼器200執行的過程互易的過程，以對位元流的經編碼的視訊資料進行解碼。例如，視訊解碼器300可以使用CABAC以與視訊編碼器200的CABAC編碼過程基本上類似的、但是互易的方式來對用於位元流的語法元素的值進行解碼。語法元素可以定義用於將圖片分割為CTU、以及根據對應的分割結構(諸如QTBT結構)對每個CTU進行分割以定義CTU的CU的分割資訊。語法元素還可以定義用於視訊資料的區塊(例如，CU)的預測和殘差資訊。In general, video decoder 300 performs processes reciprocal to those performed by video encoder 200 to decode the encoded video data of the bitstream. For example, video decoder 300 may use CABAC to decode values for syntax elements of a bitstream in a substantially similar, but reciprocal, manner to the CABAC encoding process of video encoder 200 . The syntax elements may define partition information for partitioning a picture into CTUs and partitioning each CTU according to a corresponding partition structure (such as a QTBT structure) to define the CUs of the CTUs. The syntax elements may also define prediction and residual information for blocks (eg, CUs) of video data.

殘差資訊可以由例如經量化的變換係數來表示。視訊解碼器300可以對區塊的經量化的變換係數進行逆量化和逆變換以重新產生用於該區塊的殘差區塊。視訊解碼器300使用經訊號通知的預測模式(幀內預測或幀間預測)和相關的預測資訊(例如，用於幀間預測的運動資訊)來形成用於該區塊的預測區塊。視訊解碼器300然後可以對預測區塊和殘差區塊(在逐個樣本的基礎上)進行組合以重新產生原始區塊。視訊解碼器300可以執行額外處理，諸如執行去區塊過程以減少沿著區塊的邊界的視覺偽影。The residual information may be represented by, for example, quantized transform coefficients. Video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of a block to regenerate a residual block for the block. Video decoder 300 uses the signaled prediction mode (intra prediction or inter prediction) and associated prediction information (eg, motion information for inter prediction) to form a predicted block for the block. Video decoder 300 may then combine the predicted block and the residual block (on a sample-by-sample basis) to regenerate the original block. Video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along the boundaries of blocks.

概括而言，本揭露內容可能涉及“用訊號通知”某些資訊(諸如語法元素)。術語“用訊號通知”通常可以指代對用於語法元素的值和/或用於對經編碼的視訊資料進行解碼的其它資料的傳送。也就是說，視訊編碼器200可以在位元流中用訊號通知用於語法元素的值。通常，發信指代在位元流中生成值。如上所述，源設備102可以基本上即時地或不即時地(諸如當將語法元素儲存到儲存設備112以供目的地設備116稍後取回時可能發生)將位元流傳輸到目的地設備116。In general terms, this disclosure may involve "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the transmission of values for syntax elements and/or other data used to decode encoded video data. That is, video encoder 200 may signal the values for the syntax elements in the bitstream. In general, signaling refers to generating a value in a bitstream. As noted above, source device 102 may transmit the bitstream to the destination device substantially instantaneously or not (such as may occur when syntax elements are stored to storage device 112 for later retrieval by destination device 116). 116.

概括而言，本揭露內容描述了用於對視訊資料進行寫碼的技術。具體而言，本揭露內容描述了用於對SEI訊息進行解碼的技術。本揭露內容的SEI訊息可以包括指示圖片的方位的語法元素。在另一示例中，SEI訊息可以包括指示圖片品質度量的語法元素。視訊解碼器或其它設備可以對SEI訊息進行解碼，並且根據SEI訊息來處理視訊資料的圖片。In summary, this disclosure describes techniques for encoding video data. In particular, this disclosure describes techniques for decoding SEI messages. SEI messages of the present disclosure may include syntax elements indicating the orientation of a picture. In another example, the SEI message may include a syntax element indicating a picture quality metric. A video decoder or other devices can decode the SEI information and process the pictures of the video data according to the SEI information.

通用補充增強資訊(versatile supplemental enhancement information, VSEI)標準(例如，ITU-T H.274和ISO/IEC 23002-7)規定視訊可用性資訊(video usability information, VUI)訊息以及與VVC位元流一起使用的SEI訊息中的一些SEI訊息。SEI訊息使得視訊編碼器200能夠在位元流中包括元資料，該元資料對於對輸出圖片的樣本值的正確解碼不是必需的，但是可以用於各種其它目的。視訊編碼器200可以被配置為在存取單元中包括任何數量的SEI網路抽象層(NAL)單元，並且每個SEI NAL單元可以包括一個或多個SEI訊息。使用VVC的規範和系統可以指定編碼器來生成特定SEI訊息，或者可以定義對所接收的SEI訊息的特定類型的特定處理。Versatile supplemental enhancement information (VSEI) standards (e.g., ITU-T H.274 and ISO/IEC 23002-7) specify video usability information (VUI) messages and use with VVC bitstreams Some of the SEI messages in the SEI messages. SEI messages enable video encoder 200 to include metadata in the bitstream that is not necessary for correct decoding of the sample values of the output picture, but can be used for various other purposes. Video encoder 200 may be configured to include any number of SEI network abstraction layer (NAL) units in an access unit, and each SEI NAL unit may include one or more SEI messages. Specifications and systems using VVC may specify encoders to generate specific SEI messages, or may define specific processing of specific types of received SEI messages.

以下表1規定了顯示方位SEI訊息，以向解碼器(例如，視訊解碼器300)通知被推薦在顯示之前應用於經裁剪的解碼圖片的變換：ISO/IEC JTC 1/SC 29/WG 11 N 18277，“Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 2: High Efficiency Video Coding”，2019(“HEVC”)。在下面的表1中示出了HEVC的顯示方位SEI訊息的語法結構。The following Table 1 specifies the display orientation SEI message to inform the decoder (e.g., video decoder 300) of the transform recommended to be applied to the cropped decoded picture before display: ISO/IEC JTC 1/SC 29/WG 11 N 18277, "Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 2: High Efficiency Video Coding", 2019 ("HEVC"). The syntax structure of the display position SEI message of HEVC is shown in Table 1 below.

表1 顯示方位SEI訊息語法 display_orientation( payloadSize ) { 描述符      display_orientation_cancel_flag u(1)      if( !display_orientation_cancel_flag ) {             hor_flip u(1)           ver_flip u(1)           anticlockwise_rotation u(16)           display_orientation_persistence_flag u(1)      }   }   Table 1 Show position SEI message syntax display_orientation( payloadSize ) { Descriptor display_orientation_cancel_flag u(1) if( !display_orientation_cancel_flag ) { hor_flip u(1) ver_flip u(1) anticlockwise_rotation u(16) display_orientation_persistence_flag u(1) } }

如在表1中可以看出，HEVC的顯示方位SEI訊息允許對水平翻轉(hor_flip)、垂直翻轉(ver_flip)和逆時針旋轉(anticlockwise_rotation)變換的指示。As can be seen in Table 1, the HEVC display orientation SEI message allows indication of horizontal flip (hor_flip), vertical flip (ver_flip) and anticlockwise rotation (anticlockwise_rotation) transformations.

3GPP在以下文檔中規定了視訊方位的協調(coordination of video orientation, CVO)：技術規範(technical specification, TS)26.114，“IP Multimedia Subsystem (IMS); Multimedia telephony; Media handling and interaction”，2021。CVO將在發送方側(例如，在源設備102處)捕獲的圖像的當前方位用訊號通知給接收機(例如，目的地設備116)，以進行適當的渲染和顯示。用於較低旋轉精細度的CVO資訊是以如下格式的字節來攜帶的，以支持水平翻轉和90度旋轉： 位元數量 7 6 5 4 3 2 1 0(LSB) 定義 0 0 0 0 C F R1 R0 LSB代表最低有效位。 3GPP specifies the coordination of video orientation (CVO) in the following document: Technical Specification (TS) 26.114, "IP Multimedia Subsystem (IMS); Multimedia telephony; Media handling and interaction", 2021. The CVO signals the current location of the image captured at the sender side (eg, at source device 102 ) to the receiver (eg, destination device 116 ) for appropriate rendering and display. CVO information for lower granularity of rotation is carried in bytes in the following format to support horizontal flip and 90 degree rotation: number of bits 7 6 5 4 3 2 1 0(LSB) definition 0 0 0 0 C f R1 R0 LSB stands for Least Significant Bit.

用於較高旋轉精細度的CVO資訊是以如下格式的字節來攜帶的： 位元數量 7 6 5 4 3 2 1 0(LSB) 定義 R5 R4 R3 R2 C F R1 R0 CVO information for higher granularity of rotation is carried in bytes in the following format: number of bits 7 6 5 4 3 2 1 0(LSB) definition R5 R4 R3 R2 C f R1 R0

VSEI標準的一些當前示例不支持任何方位元資料。HEVC顯示方位SEI訊息不考慮幀包裝情況，其中，旋轉應當應用於每個組成圖片而不是整個圖片。圖2示出了經幀包裝的圖片的顯示旋轉的示例，其中，每個組成圖片應當分別被旋轉。如圖2所示，圖片150包括兩個組成圖片(例如，用於立體視訊的左視圖圖片和右視圖圖片)。使用本揭露內容的技術，視訊編碼器200可以發送包括變換類型語法元素的碼和SEI訊息，該變換類型語法元素指導視訊解碼器300對組成圖片中的每個組成圖片執行旋轉變換以實施經變換的圖片152。Some current examples of the VSEI standard do not support any location metadata. HEVC display orientation SEI messages do not take frame packing into account, where rotation should be applied to each constituent picture rather than the entire picture. Fig. 2 shows an example of display rotation of frame-packed pictures, where each constituent picture should be rotated separately. As shown in FIG. 2 , the picture 150 includes two constituent pictures (eg, a left-view picture and a right-view picture for stereoscopic video). Using the techniques of this disclosure, video encoder 200 can send code and SEI messages that include transform-type syntax elements that instruct video decoder 300 to perform a rotational transform on each of the constituent pictures to implement the transformed Image 152.

示例VSEI按區域包裝(region-wise packing, RWP)SEI訊息提供了實施將經裁剪的解碼圖片的顏色樣本重新映射到經投影的圖片上的資訊。然而，當全向視訊投影被指示要應用於圖片時，使用RWP SEI訊息。在適用於圖片的寫碼層視訊序列(coded layer video sequence, CLVS)中不應存在具有等於0的rwp_cancel_flag的RWP SEI訊息。An example VSEI region-wise packing (RWP) SEI message provides information to implement the remapping of the cropped decoded picture's color samples onto the projected picture. However, the RWP SEI message is used when omnidirectional video projection is indicated to be applied to the picture. There shall not be RWP SEI messages with rwp_cancel_flag equal to 0 in the coded layer video sequence (CLVS) applicable to pictures.

圖片品質度量用於評估圖片品質和寫碼性能。以下文檔規定了媒體的定時元資料度量(諸如峰值訊噪比(peak signal-to-noise ratio, PSNR)、結構相似性索引度量(structural similary index measure, SSIM)、視訊品質度量(video quality metric, VQM)、以及ISOBMFF(ISO/IEC基本媒體檔案格式)中的平均意見分數(mean opinion score, MOS))的攜帶：ISO/IEC 23001-10，“Information technology — MPEG systems technologies — Part 10: Carriage of Timed Metadata Metrics of Media in ISO Base Media File Format”，2015。還在以下文檔中規定了圖片品質相關排序以促進品質相關視埠切換和沉浸式媒體度量：OMAF，ISO/IEC JTC1/SC29/WG11 N19042，“Text of ISO/IEC DIS 23090-2 2nd edition OMAF”，2020；以及沉浸式媒體度量(IMM)，ISO/IEC JTC1/SC29/WG3 N0073，“IS of ISO/IEC 23090-6 Immersive Media Metrics”，2020。一些圖片品質度量(諸如PSNR和SSIM)可以僅在編碼器側獲得。攜帶此類資訊的SEI訊息能夠向系統應用提供相關資訊。Image quality metrics are used to evaluate image quality and coding performance. The following documents specify timing metadata metrics for media (such as peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), video quality metric, VQM), and the carrying of the mean opinion score (mean opinion score, MOS) in ISOBMFF (ISO/IEC Basic Media File Format): ISO/IEC 23001-10, "Information technology — MPEG systems technologies — Part 10: Carriage of Timed Metadata Metrics of Media in ISO Base Media File Format”, 2015. Image quality-dependent ordering is also specified in the following document to facilitate quality-dependent viewport switching and immersive media metrics: OMAF, ISO/IEC JTC1/SC29/WG11 N19042, "Text of ISO/IEC DIS 23090-2 2nd edition OMAF" , 2020; and Immersive Media Metrics (IMM), ISO/IEC JTC1/SC29/WG3 N0073, "IS of ISO/IEC 23090-6 Immersive Media Metrics", 2020. Some picture quality metrics such as PSNR and SSIM can only be obtained at the encoder side. SEI messages carrying such information can provide relevant information to system applications.

圖片方位SEI訊息Image Orientation SEI Information

根據本揭露內容的一個示例，視訊編碼器200被配置為生成圖片方位SEI訊息並且將其用訊號通知，圖片方位SEI訊息包括在下面的表2中所示的一個或多個語法元素。具體而言，視訊編碼器200可以被配置為生成和編碼變換類型語法元素(例如，por_transform_type)，其中，變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。視訊編碼器200還可以被配置為生成和編碼在表2中列出的其它語法元素和旗標中的一者或多者。視訊解碼器300可以被配置為接收圖片方位SEI訊息，並且可以根據其中包含的語法元素來處理和/或顯示圖片。例如，視訊解碼器300可以被配置為將由變換類型語法元素指示的變換應用於經解碼的圖片。According to an example of the present disclosure, video encoder 200 is configured to generate and signal a picture orientation SEI message, which includes one or more syntax elements shown in Table 2 below. Specifically, video encoder 200 may be configured to generate and encode a transform type syntax element (eg, por_transform_type), wherein the transform type syntax element indicates a transform to be applied to a picture from among a plurality of transforms. Video encoder 200 may also be configured to generate and encode one or more of the other syntax elements and flags listed in Table 2. Video decoder 300 may be configured to receive the picture orientation SEI message, and may process and/or display the picture according to the syntax elements contained therein. For example, video decoder 300 may be configured to apply a transform indicated by a transform-type syntax element to the decoded picture.

表2 圖片方位SEI訊息語法 picture_orientation( payloadSize ) { 描述符         por_cancel_flag u(1)         if( !por_cancel_flag ) {                por_persistence_flag u(1)              por_constituent_picture_matching_flag u(1)              por_transform_type u(5)         }   }   Table 2 Picture orientation SEI message syntax picture_orientation( payloadSize ) { Descriptor por_cancel_flag u(1) if( !por_cancel_flag ) { por_persistence_flag u(1) por_constituent_picture_matching_flag u(1) por_transform_type u(5) } }

通常，圖片方位(POR)SEI訊息提供用於向視訊解碼器300通知被推薦在顯示之前應用於經解碼的圖片的變換的資訊。在一些示例中，經解碼的圖片可以是經裁剪的圖片。In general, a picture orientation (POR) SEI message provides information for informing the video decoder 300 of a transform that is recommended to be applied to a decoded picture before display. In some examples, the decoded picture may be a cropped picture.

語法元素por_cancel_flag的值等於1指示當前SEI訊息按輸出順序取消任何先前POR SEI訊息的持續性。語法元素por_cancel_flag的值等於0指示隨後是POR資訊。A value of 1 for the syntax element por_cancel_flag indicates that the current SEI message cancels the continuation of any previous POR SEI messages in output order. A value equal to 0 for the syntax element por_cancel_flag indicates that POR information follows.

語法元素por_persistence_flag的值指定用於當前層的POR SEI訊息的持續性。The value of the syntax element por_persistence_flag specifies the persistence of the POR SEI message for the current layer.

語法元素por_persistence_flag的值等於0指定POR SEI訊息僅適用於當前經解碼的圖片。A value of the syntax element por_persistence_flag equal to 0 specifies that the POR SEI message applies only to the currently decoded picture.

語法元素por_persistence_flag的值等於1指定POR SEI訊息適用於當前經解碼的圖片，並且針對當前層的所有後續圖片按輸出順序持續，直到以下條件中的一個或多個條件為真為止： –    當前層的新CLV開始。 –    位元流結束。 –    輸出在與POR SEI訊息相關聯的存取單元(access unit, AU)中的當前層中的圖片，其按輸出順序跟隨當前圖片。 A value of the syntax element por_persistence_flag equal to 1 specifies that the POR SEI message applies to the currently decoded picture and persists in output order for all subsequent pictures of the current layer until one or more of the following conditions are true: – A new CLV for the current layer starts. – End of bitstream. – Output the picture in the current layer in the access unit (AU) associated with the POR SEI message, which follows the current picture in output order.

語法元素por_constituent_picture_matching_flag的值等於1指定該SEI訊息單獨適用於每個組成圖片，並且立體幀包裝格式由幀包裝佈置SEI訊息指示。語法元素por_component_picture_matching_flag的值等於0指定該SEI訊息適用於經裁剪的解碼圖片。A value of the syntax element por_constituent_picture_matching_flag equal to 1 specifies that this SEI message applies to each constituent picture individually, and the stereoscopic frame packing format is indicated by the frame packing arrangement SEI message. A value of the syntax element por_component_picture_matching_flag equal to 0 specifies that this SEI message applies to cropped decoded pictures.

當以下條件中的任一條件為真時，語法元素por_constituent_picture_matching_flag的值應等於0： –    StereoFlag等於0。 –    StereoFlag等於1並且fp_arrangement_type等於5。 The value of the syntax element por_constituent_picture_matching_flag shall be equal to 0 when any of the following conditions is true: – StereoFlag equal to 0. – StereoFlag equal to 1 and fp_arrangement_type equal to 5.

StereoFlag的值等於0指示不存在適用於圖片的具有等於0的fp_arrangement_cancel_flag的幀包裝佈置SEI訊息。StereoFlag的值等於1指示相關聯的圖片是幀包裝圖片。A value of StereoFlag equal to 0 indicates that there is no frame packing arrangement SEI message with fp_arrangement_cancel_flag equal to 0 applicable to the picture. A value of StereoFlag equal to 1 indicates that the associated picture is a frame packed picture.

語法元素fp_arrangement_type的值等於5指示按輸出順序輸出的經裁剪的解碼圖片的分量平面形成交替的第一和第二組成幀的時間交織。A value of the syntax element fp_arrangement_type equal to 5 indicates that the component planes of the cropped decoded picture output in output order form temporal interleaving of alternating first and second constituent frames.

語法元素por_transform_type的值指定可以應用於圖片的變換(例如，旋轉、鏡像或旋轉和鏡像的組合)。注意的是，在一些示例中，鏡像可以被稱為翻轉。當由por_transform_type指示的變換指定旋轉和鏡像兩者時，視訊解碼器300可以被配置為在應用鏡像之前應用旋轉變換，或者反之亦然。在下面的表3中指定了por_transform_type的示例值。在一個示例中，por_transform_type從8到31的值被預留以供ITU-T|ISO/IEC未來使用。The value of the syntax element por_transform_type specifies a transformation (eg, rotation, mirroring, or a combination of rotation and mirroring) that may be applied to the picture. Note that mirroring may be referred to as flipping in some examples. When the transform indicated by por_transform_type specifies both rotation and mirroring, video decoder 300 may be configured to apply the rotation transform before applying mirroring, or vice versa. Example values for por_transform_type are specified in Table 3 below. In one example, por_transform_type values from 8 to 31 are reserved for future use by ITU-T|ISO/IEC.

表3 por_transform_type值 值 描述 0 無變換 1 水平鏡像 2 旋轉180度(逆時針) 3 在水平鏡像之前旋轉180度(逆時針) 4 在水平鏡像之前旋轉90度(逆時針) 5 旋轉90度(逆時針) 6 在水平鏡像之前旋轉270度(逆時針) 7 旋轉270度(逆時針) 8..31 預留 Table 3 por_transform_type value value describe 0 no transformation 1 horizontal mirror 2 Rotate 180 degrees (counterclockwise) 3 Rotate 180 degrees (counterclockwise) before mirroring horizontally 4 Rotate 90 degrees (counterclockwise) before mirroring horizontally 5 Rotate 90 degrees (counterclockwise) 6 Rotate 270 degrees (counterclockwise) before mirroring horizontally 7 Rotate 270 degrees (counterclockwise) 8..31 reserve

表3的具體數值僅是一個示例。在其它示例中，可以指定更多或更少的變換類型。此外，變換可以按與在表3中所示的順序不同的順序來指定。The specific numerical values in Table 3 are just an example. In other examples, more or fewer transform types may be specified. Furthermore, transformations may be specified in an order different from that shown in Table 3.

圖3是示出示例變換類型的概念圖。在表3的示例中，當變換語法元素具有為0的值時，視訊解碼器300可以不應用變換。圖3示出了針對其未應用變換的原始圖片160。圖3中的其它變換類型將參照原始圖片160來示出。當變換語法元素具有為1的值時，視訊解碼器300可以對原始圖片160應用水平鏡像變換以獲得圖片162。水平鏡像還可以被稱為水平翻轉。當變換語法元素具有為2的值時，視訊解碼器300可以對原始圖片160應用180度逆時針旋轉變換以獲得圖片164。當變換語法元素具有為3的值時，視訊解碼器300可以對原始圖片160應用180度逆時針旋轉變換，隨後是水平鏡像變換，以獲得圖片166。FIG. 3 is a conceptual diagram illustrating example transformation types. In the example of Table 3, when the transform syntax element has a value of 0, the video decoder 300 may not apply the transform. FIG. 3 shows an original picture 160 to which no transformation has been applied. Other transform types in FIG. 3 will be shown with reference to the original picture 160 . When the transform syntax element has a value of 1, the video decoder 300 may apply a horizontal mirror transform to the original picture 160 to obtain the picture 162 . Horizontal mirroring may also be called horizontal flipping. When the transform syntax element has a value of 2, the video decoder 300 may apply a 180-degree counterclockwise rotation transform to the original picture 160 to obtain the picture 164 . When the transform syntax element has a value of 3, the video decoder 300 may apply a 180 degree counterclockwise rotation transform to the original picture 160 followed by a horizontal mirror transform to obtain the picture 166 .

當變換語法元素具有為4的值時，視訊解碼器300可以對原始圖片160應用90度逆時針旋轉變換，隨後是水平鏡像變換，以獲得圖片168。當變換語法元素具有為5的值時，視訊解碼器300可以對原始圖片160應用90度逆時針旋轉變換以獲得圖片170。當變換語法元素具有為6的值時，視訊解碼器300可以對原始圖片160應用270度逆時針旋轉變換，隨後是水平鏡像變換，以獲得圖片172。當變換語法元素具有為7的值時，視訊解碼器300可以對原始圖片160應用270度逆時針旋轉變換以獲得圖片174。When the transform syntax element has a value of 4, the video decoder 300 may apply a 90 degree counterclockwise rotation transform to the original picture 160 followed by a horizontal mirror transform to obtain the picture 168 . When the transform syntax element has a value of 5, the video decoder 300 may apply a 90-degree counterclockwise rotation transform to the original picture 160 to obtain the picture 170 . When the transform syntax element has a value of 6, the video decoder 300 may apply a 270 degree counterclockwise rotation transform to the original picture 160 followed by a horizontal mirror transform to obtain the picture 172 . When the transform syntax element has a value of 7, the video decoder 300 may apply a 270-degree counterclockwise rotation transform to the original picture 160 to obtain the picture 174 .

圖4是示出用於對圖片方位補充增強資訊訊息進行寫碼的示例過程的流程圖。圖4示出了本揭露內容的編碼和解碼過程兩者。如圖4所示，編碼處理可以由包括視訊編碼器200的源設備102來執行。解碼過程可以由包括視訊解碼器300的目的地設備116來執行。4 is a flowchart illustrating an example process for encoding picture orientation supplemental augmentation information messages. Figure 4 shows both the encoding and decoding process of the present disclosure. As shown in FIG. 4 , the encoding process may be performed by a source device 102 including a video encoder 200 . The decoding process may be performed by the destination device 116 including the video decoder 300 .

在本揭露內容的一個示例中，源設備102可以被配置為接收圖片(400)。源設備102還可以被配置為(例如，使用視訊編碼器200)對圖片進行編碼並且向目的地設備116發送經編碼的視訊位元流。源設備102還可以被配置為確定圖片的推薦變換類型(402)。推薦變換類型可以是來自在多個變換類型當中的變換類型。源設備102還可以被配置為對包括變換類型語法元素的圖片方位訊息進行編碼，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換(404)。In one example of the present disclosure, source device 102 may be configured to receive a picture (400). Source device 102 may also be configured to encode the picture (eg, using video encoder 200 ) and send the encoded video bitstream to destination device 116 . Source device 102 may also be configured to determine a recommended transformation type for the picture (402). The recommended transformation type may be a transformation type from among a plurality of transformation types. The source device 102 may also be configured to encode the picture orientation information including a transform type syntax element indicating a transform from among the plurality of transforms to be applied to the picture (404).

目的地設備116可以被配置為接收圖片(410)。目的地設備116還可以被配置為(例如，使用視訊解碼器300)對圖片進行解碼。目的地設備116還可以對包括變換類型語法元素的圖片方位訊息進行解碼，其中，該變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換(412)。目的地設備116還可以被配置為根據變換類型語法元素來對圖片應用變換，以形成經變換的圖片(414)，並且顯示經變換的圖片(416)。Destination device 116 may be configured to receive the picture (410). Destination device 116 may also be configured to decode the picture (eg, using video decoder 300). Destination device 116 may also decode the picture orientation information including a transform type syntax element indicating a transform from among the plurality of transforms to be applied to the picture (412). Destination device 116 may also be configured to apply a transform to the picture according to the transform-type syntax element to form a transformed picture (414), and to display the transformed picture (416).

在本揭露內容的一個示例中，圖片方位訊息包括圖片方位sSEI訊息。在另一示例中，圖片方位訊息包括圖片方位開放位元流單元(open bitstream unit, OBU)。In an example of the present disclosure, the picture orientation information includes picture orientation sSEI information. In another example, the picture orientation information includes a picture orientation open bitstream unit (OBU).

如上所述，多個變換包括旋轉變換、鏡像變換或者旋轉變換和鏡像變換的組合中的兩者或更多者。在更具體的示例中，多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括180度逆時針變換，隨後是水平鏡像變換；第四變換，其包括90度逆時針變換，隨後是水平鏡像變換；第五變換，其包括90度逆時針變換；第六變換，其包括270度逆時針變換，隨後是水平鏡像變換；以及第七變換，其包括270度逆時針變換。在另外的示例中，變換類型語法元素還包括指示將不應用變換的值。As noted above, the plurality of transformations includes two or more of a rotation transformation, a mirror transformation, or a combination of rotation and mirror transformations. In a more specific example, the plurality of transformations includes: a first transformation that includes a horizontal mirror transformation; a second transformation that includes a 180-degree counterclockwise rotation transformation; a third transformation that includes a 180-degree counterclockwise transformation followed by a horizontal Mirror transformation; fourth transformation, which consists of a 90 degree counterclockwise transformation followed by a horizontal mirror transformation; fifth transformation, which consists of a 90 degree counterclockwise transformation; sixth transformation, which consists of a 270 degree counterclockwise transformation followed by a horizontal mirror transformation ; and a seventh transformation comprising a 270 degree counterclockwise transformation. In further examples, the transform-type syntax element also includes a value indicating that no transform will be applied.

在一些示例中，視訊編碼器200可以在SEI訊息中用訊號通知高精細度的旋轉。高精細度的旋轉和鏡像可以適用於每個組成圖片。通常，高精細度的旋轉可以指示以相對小的間隔旋轉的程度。例如，高精細度的旋轉可以包括以小於90度的角度旋轉圖片。在每個組成圖片可以不同地旋轉的情況下，視訊編碼器200可以在SEI訊息或其它元資料類型中指定單獨的方位變換類型或高精細度的旋轉，每一者應用於一個組成圖片。In some examples, video encoder 200 may signal high-resolution rotation in SEI messages. High-resolution rotation and mirroring can be applied to each component image. In general, a high degree of finesse of rotation can indicate the degree of rotation at relatively small intervals. For example, a high-definition rotation may include rotating a picture by an angle of less than 90 degrees. In cases where each constituent picture may be rotated differently, video encoder 200 may specify in the SEI message or other metadata type a separate orientation transformation type or a high-resolution rotation, each applied to a constituent picture.

在一些示例中，視訊編碼器200可以在CVO發信中指定組成圖片匹配旗標。組成圖片匹配旗標可以指示應用於每個組成圖片的旋轉精細度。In some examples, video encoder 200 may specify a constituent picture match flag in the CVO signaling. The component picture matching flags may indicate the granularity of rotation applied to each component picture.

圖像品質度量Image Quality Metrics SEISEI 訊息message

根據本揭露內容的另一示例，視訊編碼器200被配置為生成包括在下面的表4中所示的語法元素中的一個或多個語法元素的圖片品質度量訊息(例如，SEI訊息和/或其它封包化結構)並且將其用訊號通知。視訊解碼器300被配置為接收圖片品質度量SEI訊息，並且可以根據其中包含的語法元素來處理和/或顯示圖片。例如，目的地設備116和/或視訊解碼器300可以被配置為根據在圖片品質度量SEI訊息中指示的品質度量來對經解碼的圖片應用一個或多個後處理技術。示例後處理技術可以包括基於圖片品質來放大經解碼的圖片。在其它示例中，視訊解碼器300可以被配置為使用品質度量來選擇某些圖片以用於幀間預測。例如，當相同圖片的多個版本可用時，視訊解碼器300可以被配置為選擇具有最高品質度量(例如，最低訊噪比)的圖片，以用作幀間預測中的參考圖片。According to another example of the present disclosure, the video encoder 200 is configured to generate picture quality metric information (e.g., SEI information and/or other packetized structures) and signal it. The video decoder 300 is configured to receive the picture quality indicator SEI message, and may process and/or display the picture according to the syntax elements contained therein. For example, destination device 116 and/or video decoder 300 may be configured to apply one or more post-processing techniques to the decoded picture according to the quality metric indicated in the picture quality metric SEI message. Example post-processing techniques may include upscaling decoded pictures based on picture quality. In other examples, video decoder 300 may be configured to use quality metrics to select certain pictures for inter prediction. For example, when multiple versions of the same picture are available, video decoder 300 may be configured to select the picture with the highest quality metric (eg, lowest signal-to-noise ratio) for use as a reference picture in inter prediction.

表4是一個示例圖片品質度量SEI訊息。圖片品質度量SEI訊息提供用於當前經解碼的圖片的每個顏色分量的品質度量。Table 4 is an example picture quality measure SEI message. The picture quality metrics SEI message provides quality metrics for each color component of the currently decoded picture.

表4 圖片品質度量SEI訊息語法 Picture_quality_metrics( payloadSize ) { 描述符         pqm_metric_type u(7)         pqm_single_component_flag u(1)         for( cIdx = 0; cIdx ＜ ( dph_sei_single_component_flag ? 1 : 3 ); cIdx++ ){                if( pqm_sei_type = =  0 )                       pqm_psnr[ cIdx ] u(16)              else if ( pqm_sei_type = =  1 )                       pqm_ssim[ cIdx ] u(8)              else if ( pqm_sei_type == 2 )                       pqm_msssim[ cIdx ] u(8)              else if ( pqm_sei_type == 3 )                       pqm_vqm[ cIdx ] u(8)         }   }   Table 4 Picture Quality Metric SEI Message Syntax Picture_quality_metrics( payloadSize ) { Descriptor pqm_metric_type u(7) pqm_single_component_flag u(1) for( cIdx = 0; cIdx < ( dph_sei_single_component_flag ? 1 : 3 ); cIdx++ ){ if(pqm_sei_type == 0) pqm_psnr[ cIdx ] u(16) else if ( pqm_sei_type = = 1 ) pqm_ssim[ cIdx ] u(8) else if ( pqm_sei_type == 2 ) pqm_msssim[ cIdx ] u(8) else if ( pqm_sei_type == 3 ) pqm_vqm[ cIdx ] u(8) } }

語法元素pqm_metric_type的值指示與在表5中指定的分量相關聯的品質度量的類型。pqm_metric_type從4到127的值被預留以供ITU-T | ISO/IEC未來使用，並且在符合本規範的該版本的有效載荷資料中不應存在。The value of the syntax element pqm_metric_type indicates the type of quality metric associated with the components specified in Table 5. Values of pqm_metric_type from 4 to 127 are reserved for future use by ITU-T|ISO/IEC and shall not be present in payload profiles conforming to this version of this specification.

表5  對pqm_metric_type的解釋 pqm_metric_type 度量 0 PSNR 1 SSIM 2 MS-SSIM 3 VQM Table 5 Explanation of pqm_metric_type pqm_metric_type measure 0 PSNR 1 SSIM 2 MS-SSIM 3 VQM

PSNR品質度量類型是峰值訊噪比。SSIM品質度量類型是結構相似性索引。MS-SSIM品質度量是多尺度結構相似性索引。VQM品質度量類型是視訊品質度量。The PSNR quality metric type is Peak Signal to Noise Ratio. The SSIM quality metric type is a structural similarity index. The MS-SSIM quality measure is a multiscale structural similarity index. The VQM quality metric type is a video quality metric.

語法元素pqm_single_component_flag的值等於1指定與圖片品質度量SEI訊息相關聯的圖片包含單個顏色分量。語法元素pqm_ single_component_flag的值等於0指定與圖片品質度量SEI訊息相關聯的圖片包含三個顏色分量。pqm_single_component_flag的值應等於(ChromaFormatIdc = = 0)。A value equal to 1 for the syntax element pqm_single_component_flag specifies that the picture associated with the picture quality metric SEI information contains a single color component. A value of 0 for the syntax element pqm_single_component_flag specifies that the picture associated with the picture quality metric SEI message contains three color components. The value of pqm_single_component_flag should be equal to (ChromaFormatIdc == 0).

語法元素pqm_psnr[ cIdx ]的值指定PSNR的值。經解碼的圖片的顏色分量cIdx的對應PSNR按如下推導(以浮點表示)： PSNR = pqm_psnr[ cIdx ] / 100；除了對於pqm_psnr[ cIdx ]等於0，PSNR =無窮大以外 The value of the syntax element pqm_psnr[ cIdx ] specifies the value of the PSNR. The corresponding PSNR of the color component cIdx of the decoded picture is derived as follows (expressed in floating point): PSNR = pqm_psnr[ cIdx ] / 100; except for pqm_psnr[ cIdx ] equal to 0, PSNR = infinity

語法元素pqm_ssim[ cIdx ]的值指定SSIM的值。經解碼的圖片的顏色分量cIdx的對應SSIM按如下推導(以浮點表示)： SSIM = ( pqm_ssim[ cIdx ] – 127 ) / 128 The value of the syntax element pqm_ssim[ cIdx ] specifies the value of the SSIM. The corresponding SSIM for the color component cIdx of the decoded picture is derived as follows (expressed in floating point): SSIM = ( pqm_ssim[ cIdx ] – 127 ) / 128

語法元素pqm_msssim[ cIdx ]的值指定MS-SSIM的值。經解碼的圖片的顏色分量cIdx的對應MS-SSIM按如下推導(以浮點表示)： MS SSIM = ( pqm_msssim[ cIdx ] – 127 ) / 128 The value of the syntax element pqm_msssim[ cIdx ] specifies the value of MS-SSIM. The corresponding MS-SSIM for the color component cIdx of the decoded picture is derived as follows (expressed in floating point): MS SSIM = ( pqm_msssim[ cIdx ] – 127 ) / 128

語法元素pqm_vqm[ cIdx ]的值指定VQM的值。經解碼的圖片的顏色分量cIdx的對應VQM按如下推導(以浮點表示)： VQM = pqm_vqm[ cIdx ] / 50 The value of the syntax element pqm_vqm[ cIdx ] specifies the value of the VQM. The corresponding VQM of the color component cIdx of the decoded picture is derived as follows (expressed in floating point): VQM = pqm_vqm[ cIdx ] / 50

圖片品質度量SEI訊息可以攜帶其它品質相關度量，諸如視訊品質的感知評估(PEVQ)、平均意見分數(MOS)和/或其它圖片品質度量。The picture quality metrics SEI message may carry other quality related metrics such as Perceptual Evaluation of Video Quality (PEVQ), Mean Opinion Score (MOS) and/or other picture quality metrics.

在一些示例中，當圖片與立體幀包裝佈置SEI訊息相關聯時，圖片品質度量SEI訊息可以指定用於每個組成圖片的品質度量。當圖片與按區域幀包裝SEI訊息相關聯時，圖片品質度量SEI訊息可以指定用於每個區域的品質度量。可以向SEI訊息添加額外的語法元素，以指示對於每個組成圖片或每個區域是否存在圖片品質度量。In some examples, when a picture is associated with a stereoscopic frame packing arrangement SEI message, the picture quality metric SEI message may specify a quality metric for each constituent picture. When a picture is associated with a per-region frame packing SEI message, the picture quality metric SEI message may specify a quality metric for each region. Additional syntax elements can be added to the SEI message to indicate whether there is a picture quality metric for each constituent picture or each region.

在其它示例中，圖片品質度量SEI訊息可以攜帶與SEI訊息相關聯的圖片的一個或多個子圖片或感興趣區域(regions of interest, ROI)的品質度量。還可以在SEI訊息中指定指示子圖片或ROI的數量的語法元素、指示子圖片或ROI位置的語法元素和/或指示子圖片或ROI大小的語法元素。In other examples, the picture quality metric SEI message may carry quality metrics of one or more sub-pictures or regions of interest (ROIs) of the picture associated with the SEI message. A syntax element indicating the number of sub-pictures or ROIs, a syntax element indicating the location of sub-pictures or ROIs, and/or a syntax element indicating the size of sub-pictures or ROIs may also be specified in the SEI message.

額外品質度量(諸如加權PSNR(weighted PSNR, wPSNR)和加權到球形均勻PSNR(weighted-to-spherically uniform PSNR, WS-PSNR))可以被包括在SEI訊息中，以指示高動態範圍(high dynamic range, HDR)和360視訊內容的品質。Additional quality metrics (such as weighted PSNR (weighted PSNR, wPSNR) and weighted-to-spherically uniform PSNR (weighted-to-spherically uniform PSNR, WS-PSNR)) can be included in the SEI message to indicate high dynamic range (high dynamic range , HDR) and 360 video content quality.

在表6中提供了另一示例圖片度量SEI訊息格式： 表6 圖像品質度量SEI訊息語法 Picture_quality_metrics( payloadSize ) { 描述符         pqm_cnt_minus1 u(8)         for( i = 0; i ＜= pqm_cnt_minus1; i++ ) {                pqm_type[ i ] u(8)              pqm_value[ i ] u(16)         }   }   Another example picture metric SEI message format is provided in Table 6: Table 6 Picture Quality Metric SEI Message Syntax Picture_quality_metrics( payloadSize ) { Descriptor pqm_cnt_minus1 u(8) for( i = 0; i <= pqm_cnt_minus1; i++ ) { pqm_type[ i ] u(8) pqm_value[ i ] u(16) } }

上面的圖片度量SEI訊息提供了當前經解碼的圖片的品質度量。The picture metric SEI message above provides the quality metric of the currently decoded picture.

語法元素pqm_cnt_minus1加1的值指定通過SEI訊息指示的亮度分量品質度量的數量。The value of the syntax element pqm_cnt_minus1 plus 1 specifies the number of luma component quality metrics indicated by the SEI message.

語法元素pqm_type[ i ]的值指示與如在表7中指定的經解碼的圖片或視訊序列相關聯的第i品質度量類型。 表7  對pqm_type的解釋 pqm_type 度量 0 PSNR 1 wPSNR 2 WS-PSNR 3 PSNR sequence 4 wPSNR sequence 5 WS-PSNR sequence The value of the syntax element pqm_type[ i ] indicates the ith quality metric type associated with the decoded picture or video sequence as specified in Table 7. Table 7 Explanation of pqm_type pqm_type measure 0 PSNR 1 wxya 2 WS-PSNR 3 PSNR sequence 4 wPSNR sequence 5 WS-PSNR sequence

PSNR sequenc、wPSNR sequence和WS-PSNR sequence品質度量類型分別指示序列上的多個圖片的PSNR、wPSNR和WS-PSNR。 The PSNR sequencec , wPSNR sequence and WS-PSNR sequence quality metric types indicate the PSNR, wPSNR and WS-PSNR of multiple pictures on the sequence, respectively.

語法元素pqm_value[ i ]的值指定第i品質度量的值。當語法元素pqm_type的值為0時，則所儲存的16位元無符號整數pqm_value被解釋為PSNR值(以dB為單位)，如下所示(以浮點表示)，除了對於等於0的pqm_value值，PSNR等於無窮大以外。

Figure 02_image001
；其中，M是整數(例如，100)。 The value of the syntax element pqm_value[i] specifies the value of the i-th quality metric. When the value of the syntax element pqm_type is 0, the stored 16-bit unsigned integer pqm_value is interpreted as a PSNR value in dB as follows (expressed in floating point), except for pqm_value equal to 0 , PSNR is equal to infinity other than.
Figure 02_image001
; where M is an integer (for example, 100).

當語法元素pqm_type的值為1時，則所儲存的16位元無符號整數pqm_value被解釋為wPSNR值(以dB為單位)，如下所示(以浮點表示)，除了對於等於0的pqm_value值，wPSNR等於無窮大以外。

Figure 02_image003
；其中，M是整數(例如，100)。 When the value of the syntax element pqm_type is 1, the stored 16-bit unsigned integer pqm_value is interpreted as a wPSNR value (in dB) as follows (expressed in floating point), except for pqm_value values equal to 0 , wPSNR is equal to infinity other than.
Figure 02_image003
; where M is an integer (for example, 100).

當語法元素pqm_type的值為2時，則所儲存的16位元無符號整數pqm_value被解釋為WS-PSNR值(以dB為單位)，如下所示(以浮點表示)，除了對於等於0的pqm_value值，WS-PSNR等於無窮大以外。

Figure 02_image005
；其中M是整數(例如，100)。 When the value of the syntax element pqm_type is 2, the stored 16-bit unsigned integer pqm_value is interpreted as a WS-PSNR value in dB as follows (expressed in floating point), except for For pqm_value values other than WS-PSNR equal to infinity.
Figure 02_image005
; where M is an integer (eg, 100).

當語法元素pqm_type的值為3時，品質度量指示相關聯的圖片所屬的CLVS的平均亮度PSNR。16位元無符號整數pqm_value被解釋為序列級PSNR品質度量(以dB為單位)的結果，並且按如下推導(以浮點表示)，除了對於pqm_value值等於0，PSNR等於無窮大。

Figure 02_image007
；其中，M是整數(例如，100)。 When the value of the syntax element pqm_type is 3, the quality metric indicates the average luma PSNR of the CLVS to which the associated picture belongs. The 16-bit unsigned integer pqm_value is interpreted as the result of a sequence-level PSNR quality metric (in dB), and is derived (in floating point) as follows, except that for pqm_value values equal to 0, PSNR equals infinity.
Figure 02_image007
; where M is an integer (for example, 100).

當語法元素pqm_type的值為4時，品質度量指示相關聯的圖片所屬的CLVS的平均亮度加權PSNR。16位元無符號整數pqm_value被解釋為序列級wPSNR值(以dB為單位)，如下所示(以浮點表示)，除了對於pqm_value值等於0，wPSNR等於無窮大以外。

Figure 02_image009
；其中，M是整數(例如，100)。 When the value of the syntax element pqm_type is 4, the quality metric indicates the average luma-weighted PSNR of the CLVS to which the associated picture belongs. The 16-bit unsigned integer pqm_value is interpreted as a sequence-level wPSNR value (in dB) as follows (in floating point), except that for pqm_value values equal to 0, wPSNR equals infinity.
Figure 02_image009
; where M is an integer (for example, 100).

當語法元素pqm_type的值為5時，品質度量指示相關聯的圖片所屬的CLVS的平均亮度WS-PSNR。16位元無符號整數pqm_value被解釋為序列級WS-PSNR值(以dB為單位)，如下所示(以浮點表示)，除了對於pqm_value值等於0，WS-PSNR等於無窮大以外。

Figure 02_image011
；其中，M是整數(例如，100)。 When the value of the syntax element pqm_type is 5, the quality metric indicates the average luminance WS-PSNR of the CLVS to which the associated picture belongs. The 16-bit unsigned integer pqm_value is interpreted as a sequence-level WS-PSNR value (in dB) as follows (in floating point), except that for pqm_value values equal to 0, WS-PSNR is equal to infinity.
Figure 02_image011
; where M is an integer (for example, 100).

在另一示例中，可以在SEI訊息中包括額外品質度量類型，以指示適用於多個視訊幀的平均品質度量。可以在SEI訊息中指定第一語法元素，以指示在SEI訊息中指定的品質度量適用於相關聯的圖片並且針對當前層的的所有後續圖片按輸出順序持續。可以在SEI訊息中指定第二語法元素，以按輸出順序取消任何先前品質度量的持續性。In another example, an additional quality metric type may be included in the SEI message to indicate an average quality metric applicable to multiple video frames. A first syntax element may be specified in the SEI message to indicate that the quality metric specified in the SEI message applies to the associated picture and persists in output order for all subsequent pictures of the current layer. A second syntax element may be specified in the SEI message to de-persist any previous quality metrics in output order.

在另一示例中，當針對CLVS的任何圖片存在平均品質度量(諸如序列級PSNR)時，針對CLVS的第一圖片應存在相關聯的圖片品質SEI訊息。適用於相同CLVS的所有SEI訊息的平均圖片度量應具有相同的內容。In another example, when there is an average quality metric such as sequence-level PSNR for any picture of the CLVS, there should be associated picture quality SEI information for the first picture of the CLVS. The average picture metric for all SEI messages of the same CLVS should have the same content.

圖5是示出用於對品質度量補充增強資訊訊息進行寫碼的示例過程的流程圖。圖5示出了本揭露內容的編碼和解碼過程兩者。如圖5所示，編碼過程可以由包括視訊編碼器200的源設備102來執行。解碼過程可以由包括視訊解碼器300的目的地設備116來執行。5 is a flowchart illustrating an example process for coding a quality metric supplemental augmentation information message. Figure 5 shows both the encoding and decoding process of the present disclosure. As shown in FIG. 5 , the encoding process may be performed by the source device 102 including the video encoder 200 . The decoding process may be performed by the destination device 116 including the video decoder 300 .

在本揭露內容的一個示例中，源設備102可以被配置為接收圖片(500)。源設備102還可以被配置為(例如，使用視訊編碼器200)對圖片進行編碼並且向目的地設備116發送經編碼的視訊位元流。源設備102還可以被配置為確定用於圖片的品質度量(502)。源設備102還可以被配置為對包括品質度量語法元素的品質度量訊息進行編碼，其中，品質度量語法元素指示與圖片相關的品質度量的值(504)。In one example of the present disclosure, source device 102 may be configured to receive a picture (500). Source device 102 may also be configured to encode the picture (eg, using video encoder 200 ) and send the encoded video bitstream to destination device 116 . Source device 102 may also be configured to determine a quality metric for the picture (502). The source device 102 may also be configured to encode a quality metric message comprising a quality metric syntax element, wherein the quality metric syntax element indicates a value of a quality metric associated with the picture (504).

在本揭露內容的另外示例中，源設備102還可以被配置為在品質度量訊息中對品質度量類型語法元素進行編碼，其中，品質度量類型語法元素指示來自在多個類型的品質度量當中的通過品質度量語法元素指示的一類型的品質度量。在一個示例中，多個類型的品質度量包括峰值訊噪比(PSNR)。在另一示例中，多個類型的品質度量包括以下各項中的兩個更多項：峰值訊噪比(PSNR)、結構相似性索引(SSIM)、多尺度結構相似性索引(multiscale structural similarity index, MS-SSIM)、視訊品質度量(VQM)、加權PSNR(wPSNR)、加權到球形均勻PSNR(WS-PSNR)、序列PSNR、序列wPSNR或序列WS-PSNR。在上面的示例中，品質度量語法元素指示通過品質度量類型語法元素指示的品質度量的值。In a further example of the present disclosure, the source device 102 may be further configured to encode a quality metric type syntax element in the quality metric message, wherein the quality metric type syntax element indicates a pass from among multiple types of quality metrics. A type of quality metric indicated by the quality metric syntax element. In one example, the types of quality metrics include peak signal-to-noise ratio (PSNR). In another example, the multiple types of quality metrics include two more of the following: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Multiscale Structural Similarity Index (multiscale structural similarity index, MS-SSIM), Video Quality Metric (VQM), Weighted PSNR (wPSNR), Weighted to Spherically Uniform PSNR (WS-PSNR), Sequence PSNR, Sequence wPSNR or Sequence WS-PSNR. In the above example, the quality metric syntax element indicates the value of the quality metric indicated by the quality metric type syntax element.

目的地設備116可以被配置為接收圖片(510)。目的地設備116還可以被配置為(例如，使用視訊解碼器300)對圖片進行解碼。目的地設備116還可以對包括品質度量語法元素的品質度量訊息進行解碼，其中，品質度量語法元素指示與圖片相關的品質度量的值(512)。目的地設備116還可以被配置為根據品質度量的值來對圖片應用後處理技術，以形成經處理的圖片(514)，並且顯示經處理的圖片(516)。Destination device 116 may be configured to receive the picture (510). Destination device 116 may also be configured to decode the picture (eg, using video decoder 300). Destination device 116 may also decode the quality metric message including a quality metric syntax element indicating a value of a quality metric associated with the picture (512). Destination device 116 may also be configured to apply post-processing techniques to the picture according to the value of the quality metric to form a processed picture (514), and to display the processed picture (516).

在本揭露內容的另外示例中，目的地設備116還可以被配置為對品質度量訊息中的品質度量類型語法元素進行解碼，其中，品質度量類型語法元素指示來自在多個類型的品質度量當中的通過品質度量語法元素指示的一類型的品質度量。在一個示例中，多個類型的品質度量包括峰值訊噪比(PSNR)。在另一示例中，多個類型的品質度量包括以下各項中的兩個更多項：峰值訊噪比(PSNR)、結構相似性索引(SSIM)、多尺度結構相似性索引(MS-SSIM)、視訊品質度量(VQM)、加權PSNR(wPSNR)、加權到球形均勻PSNR(WS-PSNR)、序列PSNR、序列wPSNR或序列WS-PSNR。在上面的示例中，品質度量語法元素指示通過品質度量類型語法元素指示的品質度量的值。In another example of the present disclosure, the destination device 116 may also be configured to decode a quality metric type syntax element in the quality metric message, wherein the quality metric type syntax element indicates a quality metric from among multiple types of A type of quality metric indicated by a quality metric syntax element. In one example, the types of quality metrics include peak signal-to-noise ratio (PSNR). In another example, the multiple types of quality metrics include two more of the following: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Multiscale Structural Similarity Index (MS-SSIM) ), Video Quality Metric (VQM), Weighted PSNR (wPSNR), Weighted to Spherically Uniform PSNR (WS-PSNR), Sequence PSNR, Sequence wPSNR or Sequence WS-PSNR. In the above example, the quality metric syntax element indicates the value of the quality metric indicated by the quality metric type syntax element.

在本揭露內容的一個示例中，品質度量訊息包括品質度量SEI訊息。在另一示例中，品質度量訊息包括品質度量開放位元流單元(OBU)。In one example of the present disclosure, the quality metric information includes quality metric SEI information. In another example, the quality metric message includes a quality metric Open Bitstream Unit (OBU).

在本揭露內容的其它示例中，源設備102和/或目的地設備116可以被配置為對包括第二品質度量語法元素的第二品質度量訊息進行寫碼，其中，第二品質度量語法元素指示與圖片的子圖片或感興趣區域相關的第二品質度量的第二值。In other examples of the present disclosure, source device 102 and/or destination device 116 may be configured to encode a second quality metric message including a second quality metric syntax element, wherein the second quality metric syntax element indicates A second value of a second quality metric associated with a sub-picture or region of interest of the picture.

出於解釋的目的，在VVC(ITU-T H.266，正在開發中)和HEVC(ITU-T H.265)的背景下描述了上面的技術。然而，本揭露內容的技術可以由被配置為其它視訊寫碼標準和視訊寫碼格式(諸如AV1、AV1的未來版本以及AV1視訊寫碼格式的後繼者)的視訊編碼設備來執行。例如，與作為SEI訊息相反，這些訊息可以是封包化資料，諸如包括在本揭露內容中描述的元資料中的至少一些元資料的開放位元流單元(OBU)。作為一個示例，在圖片方位SEI訊息內包括的上述語法中的一些或全部語法可以被包括在圖片方位OBU內，使得圖片方位OBU包括取消旗標、持續性旗標、組成圖片匹配旗標或變換類型語法元素中的一者或多者。作為另一示例，在圖片品質度量SEI訊息內包括的上述語法中的一些或全部語法可以被包括在圖片品質度量OBU內，使得圖片品質度量OBU包括指示圖片的品質度量的一個或多個語法元素。For explanation purposes, the above techniques are described in the context of VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices configured for other video coding standards and video coding formats, such as AV1, future versions of AV1, and successors of the AV1 video coding format. For example, rather than being SEI messages, these messages may be packetized data, such as Open Bitstream Units (OBUs) that include at least some of the metadata described in this disclosure. As an example, some or all of the above syntax included in a picture orientation SEI message may be included in a picture orientation OBU such that the picture orientation OBU includes a cancellation flag, a persistence flag, a composition picture matching flag, or a transformation One or more of the type syntax elements. As another example, some or all of the syntaxes described above included in the picture quality metric SEI message may be included in the picture quality metric OBU such that the picture quality metric OBU includes one or more syntax elements indicating the quality metric of the picture .

圖6是示出可以執行本揭露內容的技術的示例視訊編碼器200的方塊圖。圖6是出於解釋的目的而提供的，並且不應當被認為是對如在本揭露內容中廣泛地舉例說明和描述的技術的限制。出於解釋的目的，本揭露內容描述了根據VVC(ITU-T H.266，正在開發)和HEVC(ITU-T H.265)的技術的視訊編碼器200。然而，本揭露內容的技術可以由被配置為其它視訊寫碼標準和視訊寫碼格式(諸如AV1以及AV1視訊寫碼格式的後繼者)的視訊編碼設備來執行。FIG. 6 is a block diagram illustrating an example video encoder 200 that may perform techniques of this disclosure. FIG. 6 is provided for purposes of explanation and should not be considered limiting of the techniques as broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes a video encoder 200 according to the techniques of VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video encoding devices configured to other video coding standards and video coding formats, such as AV1 and successors of the AV1 video coding format.

在圖6的示例中，視訊編碼器200包括視訊資料記憶體230、模式選擇單元202、殘差生成單元204、變換處理單元206、量化單元208、逆量化單元210、逆變換處理單元212、重構單元214、濾波器單元216、解碼圖片緩衝器(decoded picture buffer, DPB)218和熵編碼單元220。視訊資料記憶體230、模式選擇單元202、殘差生成單元204、變換處理單元206、量化單元208、逆量化單元210、逆變換處理單元212、重構單元214、濾波器單元216、DPB 218和熵編碼單元220中的任何一者或全部可以在一個或多個處理器中或者在處理電路中實施。例如，視訊編碼器200的單元可以被實施為一個或多個電路或邏輯元件，作為硬體電路的一部分，或者作為處理器、ASIC或FPGA的一部分。此外，視訊編碼器200可以包括額外的或替代的處理器或處理電路以執行這些功能和其它功能。In the example of FIG. 6 , the video coder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a re Structure unit 214, filter unit 216, decoded picture buffer (decoded picture buffer, DPB) 218 and entropy encoding unit 220. Video data memory 230, mode selection unit 202, residual generation unit 204, transformation processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transformation processing unit 212, reconstruction unit 214, filter unit 216, DPB 218 and Any or all of entropy encoding units 220 may be implemented in one or more processors or in processing circuitry. For example, the elements of video encoder 200 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC, or FPGA. Furthermore, video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

視訊資料記憶體230可以儲存要由視訊編碼器200的組件編碼的視訊資料。視訊編碼器200可以從例如視訊源104(圖1)接收在視訊資料記憶體230中儲存的視訊資料。DPB 218可以充當參考圖片記憶體，其儲存參考視訊資料以在由視訊編碼器200對後續視訊資料進行預測時使用。視訊資料記憶體230和DPB 218可以由各種記憶體設備中的任何一者形成，諸如動態隨機存取記憶體(dynamic random access memory, DRAM)(包括同步DRAM(synchronous DRAM, SDRAM))、磁阻RAM(magnetoresistive RAM, MRAM)、電阻性RAM(resistive RAM, RRAM)、或其它類型的記憶體設備。視訊資料記憶體230和DPB 218可以由相同的記憶體設備或單獨的記憶體設備提供。在各個示例中，視訊資料記憶體230可以與視訊編碼器200的其它組件在晶片上(如示出的)，或者相對於那些組件在晶片外。The video data memory 230 can store video data to be encoded by the components of the video encoder 200 . Video encoder 200 may receive video data stored in video data memory 230 from, for example, video source 104 (FIG. 1). The DPB 218 may act as a reference picture memory, which stores reference video data for use by the video encoder 200 in predicting subsequent video data. Video data memory 230 and DPB 218 may be formed by any one of various memory devices, such as dynamic random access memory (dynamic random access memory, DRAM) (including synchronous DRAM (synchronous DRAM, SDRAM)), magnetic resistance RAM (magnetoresistive RAM, MRAM), resistive RAM (resistive RAM, RRAM), or other types of memory devices. Video data memory 230 and DPB 218 may be provided by the same memory device or separate memory devices. In various examples, video data memory 230 may be on-die with other components of video encoder 200 (as shown), or off-die with respect to those components.

在本揭露內容中，對視訊資料記憶體230的引用不應當被解釋為限於在視訊編碼器200內部的記憶體(除非特別地描述為如此)或者在視訊編碼器200外部的記憶體(除非特別地描述為如此)。確切而言，對視訊資料記憶體230的引用應當被理解為儲存視訊編碼器200接收以用於編碼的視訊資料(例如，用於要被編碼的當前區塊的視訊資料)的參考記憶體。圖1的記憶體106還可以提供對來自視訊編碼器200的各個單元的輸出的臨時儲存。In this disclosure, references to video data memory 230 should not be construed as being limited to memory internal to video encoder 200 (unless specifically described as such) or external to video encoder 200 (unless specifically described as such). described as such). Specifically, the reference to the video data memory 230 should be understood as a reference memory for storing video data received by the video encoder 200 for encoding (eg, video data for the current block to be encoded). Memory 106 of FIG. 1 may also provide temporary storage of outputs from various units of video encoder 200 .

示出了圖6的各個單元以輔助理解由視訊編碼器200執行的操作。這些單元可以被實施為固定功能電路、可程式化電路、或其組合。固定功能電路指代提供特定功能並且關於可以被執行的操作而預先設置的電路。可程式化電路指代可以被程式化以執行各種任務並且在可以被執行的操作中提供靈活功能的電路。例如，可程式化電路可以執行軟體或韌體，軟體或韌體使得可程式化電路以通過軟體或韌體的指令定義的方式進行操作。固定功能電路可以執行軟體指令(例如，以接收參數或輸出參數)，但是固定功能電路執行的操作類型通常是不可變的。在一些示例中，這些單元中的一個或多個單元可以是不同的電路塊(固定功能或可程式化)，並且在一些示例中，這些單元中的一個或多個單元可以是積體電路。The various units of FIG. 6 are shown to aid in understanding the operations performed by the video encoder 200 . These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and is preset as to operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware. Fixed-function circuits can execute software instructions (eg, to receive parameters or output parameters), but the types of operations performed by fixed-function circuits are typically immutable. In some examples, one or more of these units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of these units may be an integrated circuit.

視訊編碼器200可以包括從可程式化電路形成的算術邏輯單元(arithmetic logic units, ALU)、基本功能單元(elementary function units, EFU)、數位電路、類比電路和/或可程式化核。在使用由可程式化電路執行的軟體來執行視訊編碼器200的操作的示例中，記憶體106(圖1)可以儲存視訊編碼器200接收並且執行的軟體的指令(例如，物件代碼)，或者在視訊編碼器200內的另一記憶體(未示出)可以儲存這樣的指令。The video encoder 200 may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits and/or programmable cores formed from programmable circuits. In examples where the operations of video encoder 200 are performed using software executed by programmable circuitry, memory 106 (FIG. 1) may store instructions (e.g., object code) for the software received and executed by video encoder 200, or Another memory (not shown) within video encoder 200 may store such instructions.

視訊資料記憶體230被配置為儲存所接收的視訊資料。視訊編碼器200可以從視訊資料記憶體230取回視訊資料的圖片，並且將視訊資料提供給殘差生成單元204和模式選擇單元202。視訊資料記憶體230中的視訊資料可以是要被編碼的原始視訊資料。The video data memory 230 is configured to store the received video data. The video encoder 200 can retrieve the picture of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202 . The video data in the video data memory 230 may be the original video data to be encoded.

模式選擇單元202包括運動估計單元222、運動補償單元224和幀內預測單元226。模式選擇單元202可以包括額外功能單元，其根據其它預測模式來執行視訊預測。作為示例，模式選擇單元202可以包括調色板單元、區塊內複製單元(其可以是運動估計單元222和/或運動補償單元224的一部分)、仿射單元、線性模型(linear model, LM)單元等。Mode select unit 202 includes motion estimation unit 222 , motion compensation unit 224 , and intra prediction unit 226 . The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and/or the motion compensation unit 224), an affine unit, a linear model (linear model, LM) unit etc.

模式選擇單元202通常協調多個編碼通路(pass)，以測試編碼參數的組合以及針對這樣的組合所得到的率失真值。編碼參數可以包括將CTU分割為CU、用於CU的預測模式、用於CU的殘差資料的變換類型、用於CU的殘差資料的量化參數等。模式選擇單元202可以最終選擇編碼參數的具有比其它測試的組合更佳的率失真值的組合。Mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and resulting rate-distortion values for such combinations. The encoding parameters may include partitioning of a CTU into CUs, prediction modes for CUs, transform types for residual data of CUs, quantization parameters for residual data of CUs, and the like. The mode selection unit 202 may finally select a combination of encoding parameters that has a better rate-distortion value than other tested combinations.

視訊編碼器200可以將從視訊資料記憶體230取回的圖片分割為一系列CTU，並且將一個或多個CTU封裝在切片內。模式選擇單元202可以根據樹結構(諸如上述MTT結構、QTBT結構、超級區塊結構或四元樹結構)來分割圖片的CTU。如上所述，視訊編碼器200可以從根據樹結構來分割CTU，形成一個或多個CU。這樣的CU通常還可以被稱為“視訊區塊”或“區塊”。The video encoder 200 can divide the picture retrieved from the video data memory 230 into a series of CTUs, and pack one or more CTUs into a slice. The mode selection unit 202 may divide the CTUs of the picture according to a tree structure (such as the above-mentioned MTT structure, QTBT structure, super block structure or quad tree structure). As mentioned above, the video encoder 200 can form one or more CUs by partitioning the CTU according to the tree structure. Such a CU may also be commonly referred to as a "video block" or "block".

通常，模式選擇單元202還控制其組件(例如，運動估計單元222、運動補償單元224和幀內預測單元226)以生成用於當前區塊(例如，當前CU，或者在HEVC中PU和TU的重疊部分)的預測區塊。對於當前區塊的幀間預測，運動估計單元222可以執行運動搜索以識別在一個或多個參考圖片(例如，在DPB 218中儲存的一個或多個先前經寫碼的圖片)中的一個或多個緊密匹配的參考區塊。具體地，運動估計單元222可以例如根據絕對差的總和(SAD)、平方差的總和(SSD)、平均絕對差(MAD)、均方差(MSD)等，來計算表示潛在參考區塊與當前區塊的相似程度的值。運動估計單元222通常可以使用在當前區塊與正被考慮的參考區塊之間的逐樣本差來執行這些計算。運動估計單元222可以識別從這些計算所得到的具有最低值的參考區塊，其指示與當前區塊最緊密匹配的參考區塊。Typically, mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra prediction unit 226) to generate Overlap) predicted blocks. For inter prediction of the current block, the motion estimation unit 222 may perform a motion search to identify one or more of the reference pictures (eg, one or more previously coded pictures stored in the DPB 218) or Multiple closely matching reference blocks. Specifically, the motion estimation unit 222 may, for example, calculate the difference between the potential reference block and the current block according to the sum of absolute difference (SAD), sum of squared difference (SSD), mean absolute difference (MAD), mean square difference (MSD), etc. The value of how similar the blocks are. Motion estimation unit 222 may typically perform these calculations using the sample-by-sample difference between the current block and the reference block being considered. Motion estimation unit 222 may identify the reference block resulting from these calculations with the lowest value, which indicates the reference block that most closely matches the current block.

運動估計單元222可以形成一個或多個運動向量(motion vectors, MV)，該運動向量限定相對於在當前圖片中的當前區塊的位置而言在參考圖片中的參考區塊的位置。然後，運動估計單元222可以將運動向量提供給運動補償單元224。例如，對於單向幀間預測，運動估計單元222可以提供單個運動向量，而對於雙向幀間預測，運動估計單元222可以提供兩個運動向量。然後，運動補償單元224可以使用運動向量來生成預測區塊。例如，運動補償單元224可以使用運動向量來取回參考區塊的資料。作為另一示例，如果運動向量具有分數樣本精度，則運動補償單元224可以根據一個或多個內插濾波器來對用於預測區塊的值進行內插。此外，對於雙向幀間預測，運動補償單元224可以取回用於由相應的運動向量標識的兩個參考區塊的資料並且例如通過逐樣本平均或加權平均來將所取回的資料進行組合。The motion estimation unit 222 may form one or more motion vectors (motion vectors, MVs) that define the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion estimation unit 222 may then provide the motion vectors to motion compensation unit 224 . For example, for uni-directional inter prediction, motion estimation unit 222 may provide a single motion vector, and for bi-directional inter prediction, motion estimation unit 222 may provide two motion vectors. Then, the motion compensation unit 224 can use the motion vector to generate the prediction block. For example, the motion compensation unit 224 can use the motion vector to retrieve the data of the reference block. As another example, if the motion vector has fractional sample precision, motion compensation unit 224 may interpolate the values for the prediction block according to one or more interpolation filters. Furthermore, for bi-directional inter prediction, the motion compensation unit 224 may retrieve data for two reference blocks identified by corresponding motion vectors and combine the retrieved data, eg, by sample-by-sample averaging or weighted averaging.

當根據AV1視訊寫碼格式進行操作時，運動估計單元222和運動補償單元224可以被配置為使用平移運動補償、仿射運動補償、重疊區塊運動補償(overlapped block motion compensation, OBMC)和/或複合幀內-幀間預測來對視訊資料的寫碼區塊(例如，亮度和色度寫碼區塊兩者)進行編碼。When operating in accordance with the AV1 video encoding format, the motion estimation unit 222 and the motion compensation unit 224 may be configured to use translational motion compensation, affine motion compensation, overlapped block motion compensation (overlapped block motion compensation, OBMC) and/or Composite intra-inter prediction is used to encode coded blocks of video data (eg, both luma and chroma coded blocks).

作為另一示例，對於幀內預測或幀內預測寫碼，幀內預測單元226可以根據與當前區塊相鄰的樣本來生成預測區塊。例如，對於方向性模式，幀內預測單元226通常可以數學上對相鄰樣本的值進行組合，並且跨當前區塊在所定義的方向上填充這些計算出的值以產生預測區塊。作為另一示例，對於DC模式，幀內預測單元226可以計算與當前區塊相鄰的樣本的平均，並且生成預測區塊以包括針對預測區塊的每個樣本的該得到的平均。As another example, for intra prediction or intra prediction coding, the intra prediction unit 226 may generate a prediction block according to samples adjacent to the current block. For example, for directional modes, intra prediction unit 226 may typically mathematically combine the values of neighboring samples and fill in these calculated values in a defined direction across the current block to produce a predicted block. As another example, for DC mode, intra prediction unit 226 may calculate an average of samples neighboring the current block, and generate a prediction block to include this resulting average for each sample of the prediction block.

當根據AV1視訊寫碼格式進行操作時，幀內預測單元226可以被配置為使用方向幀內預測、非方向幀內預測、遞歸濾波器幀內預測、色度根據亮度(chroma-from-luma, CFL)預測、區塊內複製(intra block copy, IBC)和/或調色板模式來對視訊資料的寫碼區塊(例如，亮度和色度寫碼區塊兩者)進行編碼。模式選擇單元202可以包括用於根據其它預測模式來執行視訊預測的額外功能單元。When operating in accordance with the AV1 video coding format, the intra prediction unit 226 may be configured to use directional intra prediction, non-directional intra prediction, recursive filter intra prediction, chroma-from-luma (chroma-from-luma, CFL) prediction, intra block copy (IBC) and/or palette mode to encode coded blocks of video data (eg, both luma and chroma coded blocks). The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes.

模式選擇單元202將預測區塊提供給殘差生成單元204。殘差生成單元204從視訊資料記憶體230接收當前區塊的原始的未經編碼的版本，並且從模式選擇單元202接收預測區塊。殘差生成單元204計算在當前區塊與預測區塊之間的逐樣本差。所得到的逐樣本差定義了用於當前區塊的殘差區塊。在一些示例中，殘差生成單元204還可以確定在殘差區塊中的樣本值之間的差，以使用殘差差分脈衝寫碼調變(residual differential pulse code modulation, RDPCM)來生成殘差區塊。在一些示例中，可以使用執行二進制減法的一個或多個減法器電路來形成殘差生成單元204。The mode selection unit 202 provides the predicted block to the residual generation unit 204 . The residual generation unit 204 receives the original unencoded version of the current block from the video data memory 230 and receives the predicted block from the mode selection unit 202 . The residual generation unit 204 calculates the sample-by-sample difference between the current block and the predicted block. The resulting sample-wise differences define the residual block for the current block. In some examples, the residual generation unit 204 may also determine the difference between sample values in the residual block to generate the residual using residual differential pulse code modulation (RDPCM) blocks. In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

在模式選擇單元202將CU分割為PU的示例中，每個PU可以與亮度預測單元和對應的色度預測單元相關聯。視訊編碼器200和視訊解碼器300可以支持具有各種大小的PU。如上文所指出的，CU的大小可以指代CU的亮度寫碼區塊的大小，而PU的大小可以指代PU的亮度預測單元的大小。假定特定CU的大小為2Nx2N，視訊編碼器200可以支持用於幀內預測的2Nx2N或NxN的PU大小、以及用於幀間預測的2Nx2N、2NxN、Nx2N、NxN或類似的對稱的PU大小。視訊編碼器200和視訊解碼器300還可以支持針對用於幀間預測的2NxnU、2NxnD、nLx2N和nRx2N的PU大小的非對稱分割。In examples where mode selection unit 202 partitions a CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 can support PUs with various sizes. As noted above, the size of a CU may refer to the size of the CU's luma coding block, while the size of the PU may refer to the size of the PU's luma prediction unit. Assuming that a particular CU has a size of 2Nx2N, the video encoder 200 may support PU sizes of 2Nx2N or NxN for intra prediction, and 2Nx2N, 2NxN, Nx2N, NxN, or similar symmetric PU sizes for inter prediction. Video encoder 200 and video decoder 300 may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction.

在模式選擇單元202不將CU進一步分割為PU的示例中，每個CU可以與亮度寫碼區塊和對應的色度寫碼區塊相關聯。如上文，CU的大小可以指代CU的亮度寫碼區塊的大小。視訊編碼器200和視訊解碼器300可以支持2Nx2N、2NxN 或 Nx2N 的CU大小。In examples where mode selection unit 202 does not further partition the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As above, the size of a CU may refer to the size of the CU's luma coding block. The video encoder 200 and the video decoder 300 can support a CU size of 2Nx2N, 2NxN or Nx2N.

對於其它視訊寫碼技術(諸如區塊內複製模式寫碼、仿射模式寫碼和線性模型(LM)模式寫碼)，作為一些示例，模式選擇單元202經由與寫碼技術相關聯的相應單元來生成用於正被編碼的當前區塊的預測區塊。在一些示例中(諸如調色板模式寫碼)，模式選擇單元202可以不生成預測區塊，而是替代地生成指示基於所選擇的調色板來重構區塊的方式的語法元素。在這樣的模式下，模式選擇單元202可以將這些語法元素提供給熵編碼單元220以被編碼。For other video coding techniques (such as intra-block copy mode coding, affine mode coding and linear model (LM) mode coding), as some examples, the mode selection unit 202 via the corresponding unit associated with the coding technique to generate a predicted block for the current block being encoded. In some examples (such as palette mode coding), the mode selection unit 202 may not generate a predicted block, but instead generate a syntax element indicating the way to reconstruct the block based on the selected palette. In such a mode, mode selection unit 202 may provide these syntax elements to entropy encoding unit 220 to be encoded.

如上所述，殘差生成單元204接收用於當前區塊和對應的預測區塊的視訊資料。然後，殘差生成單元204生成用於當前區塊的殘差區塊。為了生成殘差區塊，殘差生成單元204計算在預測區塊與當前區塊之間的逐樣本差。As mentioned above, the residual generation unit 204 receives video data for the current block and the corresponding predicted block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the predicted block and the current block.

變換處理單元206將一個或多個變換應用於殘差區塊，以生成變換係數的區塊(本文中被稱為“變換係數區塊”)。變換處理單元206可以將各種變換應用於殘差區塊，以形成變換係數區塊。例如，變換處理單元206可以將離散餘弦變換(DCT)、方向變換、Karhunen-Loeve變換(KLT)、或概念上類似的變換應用於殘差區塊。在一些示例中，變換處理單元206可以對殘差區塊執行多個變換，例如，初級變換和二次變換(諸如旋轉變換)。在一些示例中，變換處理單元206不對殘差區塊應用變換。Transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form a transform coefficient block. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform multiple transforms on the residual block, eg, a primary transform and a secondary transform (such as a rotation transform). In some examples, transform processing unit 206 applies no transform to the residual block.

當根據AV1進行操作時，變換處理單元206可以對殘差區塊應用一個或多個變換，以生成變換係數的區塊(本文中被稱為“變換係數區塊”)。變換處理單元206可以對殘差區塊應用各種變換，以形成變換係數區塊。例如，變換處理單元206可以應用水平/垂直變換組合，其可以包括離散餘弦變換(DCT)、非對稱離散正弦變換(asymmetric discrete sine transform, ADST)、翻轉ADST(例如，逆序ADST)和恆等變換(identity transform, IDTX)。當使用恆等變換時，在垂直方向或水平方向中的一者上跳過變換。在一些示例中，可以跳過變換處理。When operating in accordance with AV1, transform processing unit 206 may apply one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form a block of transform coefficients. For example, transform processing unit 206 may apply a combination of horizontal/vertical transforms, which may include discrete cosine transform (DCT), asymmetric discrete sine transform (ADST), inverted ADST (eg, reverse order ADST), and identity transform (identity transform, IDTX). When using an identity transformation, the transformation is skipped in one of the vertical or horizontal directions. In some examples, transform processing may be skipped.

量化單元208可以對變換係數區塊中的變換係數進行量化，以產生經量化的變換係數區塊。量化單元208可以根據與當前區塊相關聯的量化參數(quantization parameter, QP)值來對變換係數區塊的變換係數進行量化。視訊編碼器200(例如，經由模式選擇單元202)可以通過調整與CU相關聯的QP值來調整被應用於與當前區塊相關聯的變換係數區塊的量化程度。量化可能引入資訊損失，並且因此，經量化的變換係數可能具有與由變換處理單元206產生的原始變換係數相比較低的精度。Quantization unit 208 may quantize transform coefficients in a block of transform coefficients to generate a block of quantized transform coefficients. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (quantization parameter, QP) value associated with the current block. Video encoder 200 (eg, via mode selection unit 202 ) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and thus, quantized transform coefficients may have lower precision compared to the original transform coefficients generated by transform processing unit 206 .

逆量化單元210和逆變換處理單元212可以將逆量化和逆變換分別應用於經量化的變換係數區塊，以從變換係數區塊重構殘差區塊。重構單元214可以基於經重構的殘差區塊和由模式選擇單元202生成的預測區塊來產生與當前區塊相對應的重構區塊(儘管潛在地具有某種程度的失真)。例如，重構單元214可以將經重構的殘差區塊的樣本添加到來自由模式選擇單元202生成的預測區塊的對應樣本，以產生經重構的區塊。The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform to the quantized transform coefficient blocks, respectively, to reconstruct a residual block from the transform coefficient blocks. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although potentially with some degree of distortion) based on the reconstructed residual block and the predicted block generated by the mode selection unit 202 . For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit 202 to produce a reconstructed block.

濾波器單元216可以對經重構的區塊執行一個或多個濾波器操作。例如，濾波器單元216可以執行去區塊操作以減少沿著CU的邊緣的區塊效應偽影。在一些示例中，可以跳過濾波器單元216的操作。Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of a CU. In some examples, the operation of filter unit 216 may be skipped.

當根據AV1進行操作時，濾波器單元216可以對經重構的區塊執行一個或多個濾波器操作。例如，濾波器單元216可以執行去區塊操作以減少沿著CU的邊緣的區塊效應偽影。在其它示例中，濾波器單元216可以應用約束方向增強濾波器(constrained directional enhancement filter, CDEF)，其可以在去區塊之後被應用並且可以包括基於估計的邊緣方向來應用不可分離的非線性低通方向濾波器。濾波器單元216還可以包括環路復原濾波器，其在CDEF之後被應用並且可以包括可分離的對稱歸一化維納濾波器或雙自導濾波器。When operating in accordance with AV1, filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blockiness artifacts along the edges of a CU. In other examples, filter unit 216 may apply a constrained directional enhancement filter (CDEF), which may be applied after deblocking and may include applying a non-separable non-linear low pass direction filter. Filter unit 216 may also include a loop restoration filter, which is applied after CDEF and may include a separable symmetric normalized Wiener filter or a dual bootstrap filter.

視訊編碼器200將經重構的區塊儲存在DPB 218中。例如，在不執行濾波器單元216的操作的示例中，重構單元214可以將經重構的區塊儲存到DPB 218。在執行濾波器單元216的操作的示例中，濾波器單元216可以將經濾波的重構區塊儲存到DPB 218。運動估計單元222和運動補償單元224可以從DPB 218取回從經重構的(並且潛在地經濾波的)區塊形成的參考圖片，以對後續經編碼的圖片的區塊進行幀間預測。另外，幀內預測單元226可以使用當前圖片的在DPB 218中的經重構的區塊來對當前圖片中的其它區塊進行幀內預測。Video encoder 200 stores the reconstructed blocks in DPB 218 . For example, in examples where the operations of filter unit 216 are not performed, reconstruction unit 214 may store the reconstructed blocks to DPB 218 . In an example of performing the operations of filter unit 216 , filter unit 216 may store the filtered reconstructed block to DPB 218 . Motion estimation unit 222 and motion compensation unit 224 may retrieve reference pictures formed from reconstructed (and potentially filtered) blocks from DPB 218 for inter prediction of blocks of subsequently encoded pictures. In addition, intra prediction unit 226 may use the reconstructed blocks in DPB 218 of the current picture to perform intra prediction on other blocks in the current picture.

通常，熵編碼單元220可以對從視訊編碼器200的其它功能組件接收的語法元素進行熵編碼。例如，熵編碼單元220可以對來自量化單元208的經量化的變換係數區塊進行熵編碼。作為另一示例，熵編碼單元220可以對來自模式選擇單元202的預測語法元素(例如，用於幀間預測的運動資訊或用於幀內預測的幀內模式資訊)進行熵編碼。熵編碼單元220可以對作為視訊資料的另一示例的語法元素執行一個或多個熵編碼操作，以生成經熵編碼的資料。例如，熵編碼單元220可以對資料執行上下文自適應可變長度寫碼(context-adaptive variable length coding, CAVLC)操作、CABAC操作、可變到可變(variable-to-variable, V2V)長度寫碼操作、基於語法的上下文自適應二進制算術寫碼(syntax-based context-adaptive binary arithmetic coding, SBAC)操作、概率區間分割熵(Probability Interval Partitioning Entropy, PIPE)寫碼操作、指數哥倫布(Exponential-Golomb)編碼操作、或另一類型的熵編碼操作。在一些示例中，熵編碼單元220可以在其中語法元素未被熵編碼的旁路模式下操作。In general, the entropy encoding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200 . For example, entropy encoding unit 220 may entropy encode the quantized block of transform coefficients from quantization unit 208 . As another example, the entropy encoding unit 220 may entropy encode the prediction syntax elements (eg, motion information for inter prediction or intra mode information for intra prediction) from the mode selection unit 202 . The entropy coding unit 220 may perform one or more entropy coding operations on syntax elements, which are another example of video data, to generate entropy coded data. For example, the entropy encoding unit 220 can perform context-adaptive variable length coding (CAVLC) operations, CABAC operations, and variable-to-variable (variable-to-variable, V2V) length coding operations on the data. Operations, syntax-based context-adaptive binary arithmetic coding (SBAC) operations, probability interval partitioning entropy (Probability Interval Partitioning Entropy, PIPE) coding operations, Exponential-Golomb encoding operation, or another type of entropy encoding operation. In some examples, entropy encoding unit 220 may operate in a bypass mode in which syntax elements are not entropy encoded.

視訊編碼器200可以輸出位元流，其包括重構切片或圖片的區塊所需要的經熵編碼的語法元素。具體地，熵編碼單元220可以輸出位元流。Video encoder 200 may output a bitstream that includes entropy-coded syntax elements needed to reconstruct blocks of slices or pictures. Specifically, the entropy encoding unit 220 may output a bit stream.

根據AV1，熵編碼單元220可以被配置為符號到符號自適應多符號算術寫碼器。AV1中的語法元素包括N個元素的字母表，並且上下文(例如，概率模型)包括N個概率的集合。熵編碼單元220可以將概率儲存為n位元(例如，15位元)累積分佈函數(cumulative distribution functions, CDF)。熵編碼單元22可以利用基於字母表大小的更新因子來執行遞歸縮放以更新上下文。According to AV1, the entropy coding unit 220 may be configured as a symbol-to-symbol adaptive multi-symbol arithmetic coder. A syntax element in AV1 includes an alphabet of N elements, and a context (eg, a probability model) includes a set of N probabilities. The entropy encoding unit 220 may store the probabilities as n-bit (eg, 15-bit) cumulative distribution functions (cumulative distribution functions, CDF). The entropy encoding unit 22 may perform recursive scaling with an update factor based on the alphabet size to update the context.

關於區塊描述了上述操作。這樣的描述應當被理解為用於亮度寫碼區塊和/或色度寫碼區塊的操作。如上所述，在一些示例中，亮度寫碼區塊和色度寫碼區塊是CU的亮度分量和色度分量。在一些示例中，亮度寫碼區塊和色度寫碼區塊是PU的亮度分量和色度分量。The above operations have been described with respect to blocks. Such a description should be understood as being for the operation of a luma code block and/or a chroma code block. As described above, in some examples, the luma and chroma write code blocks are the luma and chroma components of the CU. In some examples, the luma write code block and the chroma write code block are the luma and chroma components of the PU.

在一些示例中，關於亮度寫碼區塊執行的操作不需要被重複用於色度寫碼區塊。作為一個示例，用於識別用於亮度寫碼區塊的運動向量(MV)和參考圖片的操作不需要被重複用於識別用於色度區塊的MV和參考圖片。確切而言，可以對用於亮度寫碼區塊的MV進行縮放以確定用於色度區塊的MV，並且參考圖片可以是相同的。作為另一示例，對於亮度寫碼區塊和色度寫碼區塊，幀內預測過程可以是相同的。In some examples, operations performed on luma write blocks need not be repeated for chroma write blocks. As one example, the operations used to identify motion vectors (MVs) and reference pictures for luma coded blocks need not be repeated for identifying MVs and reference pictures for chroma blocks. Specifically, the MV for the luma block can be scaled to determine the MV for the chroma block, and the reference picture can be the same. As another example, the intra prediction process may be the same for luma and chroma coded blocks.

根據上面討論的SEI技術，視訊編碼器200表示被配置為對視訊資料進行編碼的設備的示例，該設備包括：被配置為儲存視訊資料的記憶體；以及在電路中實施並且被配置為進行以下操作的一個或多個處理單元：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行編碼，其中，變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。視訊編碼器200還可以被配置為對包括品質度量語法元素的品質度量訊息進行編碼，其中，品質度量語法元素指示與圖片相關的品質度量的值。In accordance with the SEI techniques discussed above, video encoder 200 represents an example of a device configured to encode video data comprising: a memory configured to store video data; and implemented in circuitry and configured to: The one or more processing units of operation: receive a picture; and encode picture orientation information including a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. The video encoder 200 may also be configured to encode a quality metric message including a quality metric syntax element indicating a value of a quality metric associated with a picture.

圖7是示出可以執行本揭露內容的技術的示例視訊解碼器300的方塊圖。圖7是出於解釋的目的而提供的，並且不是對如在本揭露內容中廣泛地舉例說明和描述的技術的限制。出於解釋的目的，本揭露內容描述了根據VVC(ITU-T H.266，正在開發)和HEVC(ITU-T H.265)的技術的視訊解碼器300。然而，本揭露內容的技術可以由被配置為其它視訊寫碼標準的視訊寫碼設備來執行。FIG. 7 is a block diagram illustrating an example video decoder 300 that may perform techniques of this disclosure. FIG. 7 is provided for purposes of explanation, and not limitation of the techniques as broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes a video decoder 300 according to the techniques of VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265). However, the techniques of this disclosure may be performed by video coding devices configured to other video coding standards.

在圖7的示例中，視訊解碼器300包括寫碼圖片緩衝器(coded picture buffer, CPB)記憶體320、熵解碼單元302、預測處理單元304、逆量化單元306、逆變換處理單元308、重構單元310、濾波器單元312和解碼圖片緩衝器(DPB)134。CPB記憶體320、熵解碼單元302、預測處理單元304、逆量化單元306、逆變換處理單元308、重構單元310、濾波器單元312和DPB 134中的任何一者或全部可以在一個或多個處理器中或者在處理電路中實施。例如，視訊解碼器300的單元可以被實施為一個或多個電路或邏輯元件，作為硬體電路的一部分，或者作為處理器、ASIC或FPGA的一部分。此外，視訊解碼器300可以包括額外的或替代的處理器或處理電路以執行這些功能和其它功能。In the example of FIG. 7 , the video decoder 300 includes a coded picture buffer (coded picture buffer, CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a re- Construction unit 310, filter unit 312 and decoded picture buffer (DPB) 134. Any one or all of CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312 and DPB 134 can be in one or more implemented in a processor or in a processing circuit. For example, the elements of video decoder 300 may be implemented as one or more circuits or logic elements, as part of a hardware circuit, or as part of a processor, ASIC or FPGA. Furthermore, video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

預測處理單元304包括運動補償單元316和幀內預測單元318。預測處理單元304可以包括用於根據其它預測模式來執行預測的額外的單元。作為示例，預測處理單元304可以包括調色板單元、區塊內複製單元(其可以形成運動補償單元316的一部分)、仿射單元、線性模型(LM)單元等。在其它示例中，視訊解碼器300可以包括更多、更少或不同的功能組件。Prediction processing unit 304 includes motion compensation unit 316 and intra prediction unit 318 . Prediction processing unit 304 may include additional units for performing prediction according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit 316 ), an affine unit, a linear model (LM) unit, and the like. In other examples, video decoder 300 may include more, fewer or different functional components.

當根據AV1進行操作時，補償單元316可以被配置為使用平移運動補償、仿射運動補償、OBMC和/或複合幀間幀內預測來對視訊資料的寫碼區塊(例如，亮度和色度寫碼區塊兩者)進行解碼，如上所述。幀內預測單元318可以被配置為使用方向幀內預測、非方向幀內預測、遞歸濾波器幀內預測、CFL、區塊內複製(IBC)和/或調色板模式來對視訊資料的寫碼區塊(例如，亮度和色度寫碼區塊兩者)進行解碼。When operating in accordance with AV1, the compensation unit 316 may be configured to use translational motion compensation, affine motion compensation, OBMC, and/or complex inter-intra prediction to encode blocks of video material (e.g., luma and chroma write code blocks) to decode, as described above. The intra prediction unit 318 may be configured to write video data using directional intra prediction, non-directional intra prediction, recursive filter intra prediction, CFL, intra block copy (IBC) and/or palette mode. Code blocks (eg, both luma and chroma write code blocks) are decoded.

CPB記憶體320可以儲存要由視訊解碼器300的組件解碼的視訊資料，諸如經編碼的視訊位元流。可以例如從電腦可讀媒體110(圖1)獲得在CPB記憶體320中儲存的視訊資料。CPB記憶體320可以包括儲存來自經編碼的視訊位元流的經編碼的視訊資料(例如，語法元素)的CPB。此外，CPB記憶體320可以儲存除了經寫碼的圖片的語法元素之外的視訊資料，諸如表示來自視訊解碼器300的各個單元的輸出的臨時資料。DPB 314通常儲存經解碼的圖片，視訊解碼器300可以輸出經解碼的圖片，和/或當對經編碼的視訊位元流的後續資料或圖片進行解碼時使用經解碼的圖片作為參考視訊資料。CPB記憶體320和DPB 314可以由各種記憶體設備中的任何一者形成，諸如DRAM(包括SDRAM)、MRAM、RRAM或其它類型的記憶體設備。CPB記憶體320和DPB 314可以由相同的記憶體設備或單獨的記憶體設備來提供。在各個示例中，CPB記憶體320可以與視訊解碼器300的其它組件在晶片上，或者相對於那些組件在晶片外。CPB memory 320 may store video data to be decoded by components of video decoder 300 , such as an encoded video bitstream. Video data stored in CPB memory 320 may be obtained, for example, from computer readable medium 110 (FIG. 1). CPB memory 320 may include a CPB that stores encoded video data (eg, syntax elements) from an encoded video bitstream. In addition, the CPB memory 320 may store video data other than the syntax elements of the coded picture, such as temporary data representing outputs from various units of the video decoder 300 . DPB 314 typically stores decoded pictures that video decoder 300 may output and/or use as reference video data when decoding subsequent data or pictures in the encoded video bitstream. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as DRAM (including SDRAM), MRAM, RRAM, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, CPB memory 320 may be on-die with other components of video decoder 300, or off-die with respect to those components.

另外或替代地，在一些示例中，視訊解碼器300可以從記憶體120(圖1)取回經寫碼的視訊資料。也就是說，記憶體120可以如上文所討論地利用CPB記憶體320來儲存資料。同樣，當視訊解碼器300的一些或全部功能是用要被視訊解碼器300的處理電路執行的軟體來實施的時，記憶體120可以儲存要被視訊解碼器300執行的指令。Additionally or alternatively, in some examples, video decoder 300 may retrieve encoded video data from memory 120 (FIG. 1). That is, the memory 120 can utilize the CPB memory 320 to store data as discussed above. Likewise, the memory 120 may store instructions to be executed by the video decoder 300 when some or all of the functions of the video decoder 300 are implemented by software to be executed by the processing circuit of the video decoder 300 .

示出了圖7中示出的各個單元以輔助理解由視訊解碼器300執行的操作。這些單元可以被實施為固定功能電路、可程式化電路、或其組合。類似於圖6，固定功能電路指代提供特定功能並且關於可以被執行的操作而預先設置的電路。可程式化電路指代可以被程式化以執行各種任務並且在可以被執行的操作中提供靈活功能的電路。例如，可程式化電路可以執行軟體或韌體，軟體或韌體使得可程式化電路以通過軟體或韌體的指令定義的方式進行操作。固定功能電路可以執行軟體指令(例如，以接收參數或輸出參數)，但是固定功能電路執行的操作的類型通常是不可變的。在一些示例中，這些單元中的一個或多個單元可以是不同的電路塊(固定功能或可程式化)，並且在一些示例中，這些單元中的一個或多個單元可以是積體電路。The various units shown in FIG. 7 are shown to assist in understanding the operations performed by the video decoder 300 . These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Similar to FIG. 6 , a fixed-function circuit refers to a circuit that provides a specific function and is preset as to operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware. Fixed-function circuits may execute software instructions (eg, to receive parameters or output parameters), but the types of operations performed by fixed-function circuits are typically immutable. In some examples, one or more of these units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of these units may be an integrated circuit.

視訊解碼器300可以包括從可程式化電路形成的ALU、EFU、數位電路、類比電路和/或可程式化核。在視訊解碼器300的操作由在可程式化電路上執行的軟體執行的示例中，晶片上或晶片外記憶體可以儲存視訊解碼器300接收並且執行的軟體的指令(例如，物件代碼)。Video decoder 300 may include ALUs, EFUs, digital circuits, analog circuits and/or programmable cores formed from programmable circuits. In examples where the operations of video decoder 300 are performed by software executing on a programmable circuit, on-chip or off-chip memory may store instructions (eg, object code) for the software that video decoder 300 receives and executes.

熵解碼單元302可以從CPB接收經編碼的視訊資料，並且對視訊資料進行熵解碼以重新產生語法元素。預測處理單元304、逆量化單元306、逆變換處理單元308、重構單元310和濾波器單元312可以基於從位元流中提取的語法元素來生成經解碼的視訊資料。The entropy decoding unit 302 may receive encoded video data from the CPB, and perform entropy decoding on the video data to regenerate syntax elements. Prediction processing unit 304 , inverse quantization unit 306 , inverse transform processing unit 308 , reconstruction unit 310 , and filter unit 312 may generate decoded video data based on syntax elements extracted from the bitstream.

通常，視訊解碼器300在逐區塊的基礎上重構圖片。視訊解碼器300可以單獨地對每個區塊執行重構操作(其中，當前正在被重構(即，被解碼)的區塊可以被稱為“當前區塊”)。Typically, video decoder 300 reconstructs pictures on a block-by-block basis. The video decoder 300 may individually perform a reconstruction operation on each block (wherein, the block currently being reconstructed (ie, decoded) may be referred to as a "current block").

熵解碼單元302可以對定義經量化的變換係數區塊的經量化的變換係數的語法元素以及諸如量化參數(QP)和/或變換模式指示的變換資訊進行熵解碼。逆量化單元306可以使用與經量化的變換係數區塊相關聯的QP來確定量化程度，並且同樣地，確定供逆量化單元306應用的逆量化程度。逆量化單元306可以例如執行按位左移操作以對經量化的變換係數進行逆量化。逆量化單元306從而可以形成包括變換係數的變換係數區塊。The entropy decoding unit 302 may entropy decode syntax elements defining quantized transform coefficients of a block of quantized transform coefficients and transform information such as quantization parameters (QP) and/or transform mode indications. Inverse quantization unit 306 may use the QP associated with the block of quantized transform coefficients to determine the degree of quantization, and as such, determine the degree of inverse quantization for inverse quantization unit 306 to apply. Inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thus form a transform coefficient block comprising transform coefficients.

在逆量化單元306形成變換係數區塊之後，逆變換處理單元308可以將一個或多個逆變換應用於變換係數區塊，以生成與當前區塊相關聯的殘差區塊。例如，逆變換處理單元308可以將逆DCT、逆整數變換、逆Karhunen-Loeve變換(KLT)、逆旋轉變換、逆方向變換或另一逆變換應用於變換係數區塊。After the transform coefficient block is formed by the inverse quantization unit 306, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transform to the block of transform coefficients.

此外，預測處理單元304根據由熵解碼單元302進行熵解碼的預測資訊語法元素來生成預測區塊。例如，如果預測資訊語法元素指示當前區塊是經幀間預測的，則運動補償單元316可以生成預測區塊。在這種情況下，預測資訊語法元素可以指示在DPB 314中的要從其取回參考區塊的參考圖片、以及標識相對於在當前圖片中的當前區塊的位置而言在參考圖片中的參考區塊的位置的運動向量。運動補償單元316通常可以以與關於運動補償單元224(圖6)所描述的方式基本上類似的方式來執行幀間預測過程。In addition, the prediction processing unit 304 generates a prediction block according to the prediction information syntax element entropy-decoded by the entropy decoding unit 302 . For example, motion compensation unit 316 may generate a predicted block if the prediction information syntax element indicates that the current block is inter-predicted. In this case, the prediction information syntax element may indicate the reference picture in the DPB 314 from which the reference block is to be retrieved, and identify the location in the reference picture relative to the position of the current block in the current picture. The motion vector of the location of the reference block. Motion compensation unit 316 may generally perform the inter prediction process in a manner substantially similar to that described with respect to motion compensation unit 224 (FIG. 6).

作為另一示例，如果預測資訊語法元素指示當前區塊是經幀內預測的，則幀內預測單元318可以根據通過預測資訊語法元素指示的幀內預測模式來生成預測區塊。再次，幀內預測單元318通常可以以與關於幀內預測單元226(圖6)所描述的方式基本上類似的方式來執行幀內預測過程。幀內預測單元318可以從DPB 314取回與當前區塊相鄰的樣本的資料。As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra prediction unit 318 may generate the prediction block according to the intra prediction mode indicated by the prediction information syntax element. Again, intra-prediction unit 318 may generally perform the intra-prediction process in a manner substantially similar to that described with respect to intra-prediction unit 226 (FIG. 6). The intra prediction unit 318 may retrieve data from the DPB 314 for samples adjacent to the current block.

重構單元310可以使用預測區塊和殘差區塊來重構當前區塊。例如，重構單元310可以將殘差區塊的樣本添加到預測區塊的對應樣本來重構當前區塊。The reconstruction unit 310 may use the prediction block and the residual block to reconstruct the current block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the predicted block to reconstruct the current block.

濾波器單元312可以對經重構的區塊執行一個或多個濾波器操作。例如，濾波器單元312可以執行去區塊操作以減少沿著經重構的區塊的邊緣的區塊效應偽影。不一定在所有示例中都執行濾波器單元312的操作。Filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed blocks. The operations of filter unit 312 are not necessarily performed in all examples.

視訊解碼器300可以將經重構的區塊儲存在DPB 314中。例如，在不執行濾波器單元312的操作的示例中，重構單元310可以將經重構的區塊儲存到DPB 314。在執行濾波器單元312的操作的示例中，濾波器單元312可以將經濾波的重構區塊儲存到DPB 314。如上文所討論的，DPB 314可以將參考資訊(諸如用於幀內預測的當前圖片以及用於後續運動補償的先前經解碼的圖片的樣本)提供給預測處理單元304。此外，視訊解碼器300可以從DPB 314輸出經解碼的圖片(例如，經解碼的視訊)，以用於在諸如圖1的顯示設備118的顯示設備上的後續呈現。Video decoder 300 may store the reconstructed blocks in DPB 314 . For example, in examples where the operations of filter unit 312 are not performed, reconstruction unit 310 may store the reconstructed blocks to DPB 314 . In an example performing the operations of filter unit 312 , filter unit 312 may store the filtered reconstructed block to DPB 314 . As discussed above, DPB 314 may provide reference information, such as samples of the current picture for intra prediction and previously decoded pictures for subsequent motion compensation, to prediction processing unit 304 . Additionally, video decoder 300 may output decoded pictures (eg, decoded video) from DPB 314 for subsequent presentation on a display device, such as display device 118 of FIG. 1 .

根據上面討論的SEI技術，視訊解碼器300表示被配置為對視訊資料進行解碼的設備的示例，該設備包括：被配置為儲存視訊資料的記憶體；以及在電路中實施並且被配置為進行以下操作的一個或多個處理單元：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行解碼，其中，變換類型語法元素指示來自在多個變換當中的要應用於圖片的變換。視訊解碼器300還可以被配置為對包括品質度量語法元素的品質度量訊息進行解碼，其中，品質度量語法元素指示與圖片相關的品質度量的值。In accordance with the SEI techniques discussed above, video decoder 300 represents an example of a device configured to decode video data comprising: a memory configured to store video data; and implemented in a circuit and configured to: The one or more processing units of operation: receive a picture; and decode picture orientation information including a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. The video decoder 300 may also be configured to decode a quality metric message including a quality metric syntax element indicating a value of a quality metric associated with a picture.

圖8是示出根據本揭露內容的技術的用於對當前區塊進行編碼的示例方法的流程圖。當前區塊可以包括當前CU。儘管關於視訊編碼器200(圖1和圖6)進行了描述，但是應當理解的是，其它設備可以被配置為執行與圖8的方法類似的方法。8 is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure. A current block may include a current CU. Although described with respect to video encoder 200 ( FIGS. 1 and 6 ), it should be understood that other devices may be configured to perform a method similar to that of FIG. 8 .

在該示例中，視訊編碼器200最初預測當前區塊(350)。例如，視訊編碼器200可以形成用於當前區塊的預測區塊。然後，視訊編碼器200可以計算用於當前區塊的殘差區塊(352)。為了計算殘差區塊，視訊編碼器200可以計算在原始的未經編碼的區塊與用於當前區塊的預測區塊之間的差。然後，視訊編碼器200可以對殘差區塊進行變換以及對殘差區塊的變換係數進行量化(354)。接下來，視訊編碼器200可以掃描殘差區塊的經量化的變換係數(356)。在掃描期間或在掃描之後，視訊編碼器200可以對變換係數進行熵編碼(358)。例如，視訊編碼器200可以使用CAVLC或CABAC來對變換係數進行編碼。然後，視訊編碼器200可以輸出區塊的經熵編碼的資料(360)。In this example, video encoder 200 initially predicts the current block (350). For example, the video encoder 200 may form a predicted block for the current block. Video encoder 200 may then compute a residual block for the current block (352). To calculate the residual block, video encoder 200 may calculate the difference between the original uncoded block and the predicted block for the current block. Video encoder 200 may then transform the residual block and quantize the transform coefficients of the residual block (354). Next, video encoder 200 may scan the residual block for quantized transform coefficients (356). During the scan or after the scan, video encoder 200 may entropy encode the transform coefficients (358). For example, video encoder 200 may use CAVLC or CABAC to encode transform coefficients. Video encoder 200 may then output entropy encoded data for the block (360).

圖9是示出根據本揭露內容的技術的用於對視訊資料的當前區塊進行解碼的示例方法的流程圖。當前區塊可以包括當前CU。儘管關於視訊解碼器300(圖1和圖7)進行了描述，但是應當理解的是，其它設備可以被配置為執行與圖9的方法類似的方法。9 is a flowchart illustrating an example method for decoding a current block of video material in accordance with the techniques of this disclosure. A current block may include a current CU. Although described with respect to video decoder 300 ( FIGS. 1 and 7 ), it should be understood that other devices may be configured to perform a method similar to that of FIG. 9 .

視訊解碼器300可以接收用於當前區塊的經熵編碼的資料(諸如經熵編碼的預測資訊和用於與當前區塊相對應的殘差區塊的變換係數的經熵編碼的資料)(370)。視訊解碼器300可以對經熵編碼的資料進行熵解碼以確定用於當前區塊的預測資訊並且重新產生殘差區塊的變換係數(372)。視訊解碼器300可以例如使用如由用於當前區塊的預測資訊指示的幀內預測模式或幀間預測模式來預測當前區塊(374)，以計算用於當前區塊的預測區塊。然後，視訊解碼器300可以對所重新產生的變換係數進行逆掃描(376)，以創建經量化的變換係數的區塊。然後，視訊解碼器300可以對變換係數進行逆量化以及將逆變換應用於變換係數以產生殘差區塊(378)。最終，視訊解碼器300可以通過將預測區塊和殘差區塊進行組合來對當前區塊進行解碼(380)。The video decoder 300 may receive entropy-coded data for a current block (such as entropy-coded prediction information and entropy-coded data for transform coefficients of a residual block corresponding to the current block) ( 370). Video decoder 300 may entropy decode the entropy encoded data to determine prediction information for the current block and regenerate transform coefficients for the residual block (372). The video decoder 300 may predict the current block (374), eg, using an intra prediction mode or an inter prediction mode as indicated by the prediction information for the current block, to calculate a predicted block for the current block. Video decoder 300 may then inverse scan (376) the regenerated transform coefficients to create blocks of quantized transform coefficients. Video decoder 300 may then inverse quantize and apply the inverse transform to the transform coefficients to generate a residual block (378). Finally, the video decoder 300 may decode the current block by combining the predicted block and the residual block (380).

下文描述了本揭露內容的技術和設備的其它說明性方面。Other illustrative aspects of the techniques and devices of the disclosure are described below.

方面1A-一種處理視訊資料的方法，所述方法包括：接收圖片；以及對圖片方位補充增強資訊(SEI)訊息進行寫碼，所述圖片方位SEI訊息包括以下各項中的一項或多項：取消旗標、持續性旗標、組成圖片匹配旗標或變換類型語法元素，其中，所述變換類型語法元素指示要應用於所述圖片的旋轉或鏡像中的一者或多者。Aspect 1A - A method of processing video data, the method comprising: receiving a picture; and encoding a picture orientation Supplemental Enhancement Information (SEI) message, the picture orientation SEI message comprising one or more of the following: A cancellation flag, a persistence flag, a constituent picture matching flag, or a transform-type syntax element, wherein the transform-type syntax element indicates one or more of rotation or mirroring to be applied to the picture.

方面2A-根據方面1A所述的方法，其中，寫碼包括解碼，並且其中，所述方法還包括：根據所述圖片方位SEI訊息來處理所述圖片。Aspect 2A. The method of Aspect 1A, wherein encoding comprises decoding, and wherein the method further comprises processing the picture according to the picture orientation SEI information.

方面3A-根據方面1A或方面2A所述的方法，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 3A - The method of Aspect 1A or Aspect 2A, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面4A-根據方面1A或方面2A所述的方法，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 4A - The method of Aspect 1A or Aspect 2A, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面5A-如請求項1A所述的方法，其中，寫碼包括編碼。Aspect 5A - The method of claim 1A, wherein coding includes encoding.

方面6A-一種處理視訊資料的方法，所述方法包括：接收圖片；以及對圖片品質度量補充增強資訊(SEI)訊息進行寫碼，所述圖片品質度量SEI訊息包括指示所述圖片的品質度量的一個或多個語法元素。Aspect 6A - A method of processing video data, the method comprising: receiving a picture; and encoding a picture quality metric Supplemental Enhancement Information (SEI) message, the picture quality metric SEI message including a metric indicating a quality metric of the picture One or more syntax elements.

方面7A-根據方面6A所述的方法，其中，寫碼包括解碼，並且其中，所述方法還包括：根據所述圖片品質度量SEI訊息來處理所述圖片。Aspect 7A - The method of Aspect 6A, wherein encoding includes decoding, and wherein the method further includes processing the picture according to the picture quality metric SEI message.

方面8A-根據方面6A或方面7A所述的方法，其中，所述圖片方位訊息包括圖片品質度量補充增強資訊(SEI)訊息。Aspect 8A - The method of Aspect 6A or Aspect 7A, wherein the picture orientation information comprises a picture quality metric Supplemental Enhancement Information (SEI) message.

方面9A-根據方面6A-8A中任一項所述的方法，其中，所述圖片品質度量訊息包括一個或多個語法元素，所述一個或多個語法元素指示所述圖片的與所述圖片品質度量訊息相關聯的一個或多個子圖片或感興趣區域的品質度量。Aspect 9A - The method of any one of aspects 6A-8A, wherein the picture quality metric message includes one or more syntax elements indicating a relationship between the picture and the picture The quality metric of one or more sub-pictures or ROIs associated with the quality metric information.

方面10A-根據方面9A所述的方法，其中，所述一個或多個語法元素指示高動態範圍(HDR)或360視訊內容的品質度量。Aspect 10A - The method of Aspect 9A, wherein the one or more syntax elements indicate a quality metric for high dynamic range (HDR) or 360 video content.

方面11A-根據方面6A-10A中任一項所述的方法，其中，寫碼包括編碼。Aspect 11A - The method of any one of aspects 6A-10A, wherein coding includes encoding.

方面12A-根據方面1A-10A所述的方法的任何組合。Aspect 12A - Any combination of the methods of aspects 1A-10A.

方面13A-一種用於處理視訊資料的設備，所述設備包括用於執行根據方面1A-12A中任一項所述的方法的一個或多個部件。Aspect 13A - An apparatus for processing video material, said apparatus comprising one or more means for performing the method according to any one of aspects 1A-12A.

方面14A-根據方面13A所述的設備，其中，所述一個或多個部件包括在電路中實施的一個或多個處理器。Aspect 14A - The apparatus of Aspect 13A, wherein the one or more components comprise one or more processors implemented in circuitry.

方面15A-根據方面13A和14A中任一項所述的設備，還包括：用於儲存所述視訊資料的記憶體。Aspect 15A - The apparatus of any one of aspects 13A and 14A, further comprising memory for storing said video data.

方面16A-根據方面13A-15A中任一項所述的設備，還包括：被配置為顯示經解碼的視訊資料的顯示器。Aspect 16A - The apparatus of any one of aspects 13A-15A, further comprising a display configured to display the decoded video material.

方面17A-根據方面13A-16A中任一項所述的設備，其中，所述設備包括以下各項中的一項或多項：相機、電腦、行動設備、廣播接收機設備或機上盒。Aspect 17A - The device of any of aspects 13A-16A, wherein the device comprises one or more of: a camera, a computer, a mobile device, a broadcast receiver device, or a set top box.

方面18A-根據方面13A-17A中任一項所述的設備，其中，所述設備包括視訊解碼器。Aspect 18A - The apparatus of any one of aspects 13A-17A, wherein the apparatus comprises a video decoder.

方面19A-根據方面13A-18A中任一項所述的設備，其中，所述設備包括視訊編碼器。Aspect 19A - The apparatus of any of aspects 13A-18A, wherein the apparatus comprises a video encoder.

方面20A-一種在其上儲存有指令的電腦可讀儲存媒體，所述指令當被執行時使得一個或多個處理器執行根據方面1A-12A中任一項所述的方法。Aspect 20A - A computer readable storage medium having stored thereon instructions which when executed cause one or more processors to perform the method according to any one of aspects 1A-12A.

方面1B-一種處理視訊資料的方法，所述方法包括：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 1B - A method of processing video data, the method comprising: receiving a picture; and encoding picture orientation information including a transform type syntax element, wherein the transform type syntax element indicates a request from among a plurality of transforms The transform to apply to the image.

方面2B-根據方面1B所述的方法，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。Aspect 2B - The method of Aspect 1B, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of the rotation transformation and the mirror transformation.

方面3B-根據方面1B所述的方法，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 3B - The method of Aspect 1B, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面4B-根據方面1B所述的方法，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 4B - The method of Aspect IB, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面5B-根據方面1B所述的方法，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 5B - The method of Aspect 1B, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the The persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面6B-根據方面1B所述的方法，其中，寫碼包括解碼，並且其中，所述方法還包括：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 6B - The method of Aspect 1B, wherein encoding includes decoding, and wherein the method further comprises applying the transform to the picture according to the transform-type syntax element to form a transformed picture ; and displaying the transformed picture.

方面7B-根據方面1B所述的方法，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 7B - The method of aspect 1B, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面8B-根據方面1B所述的方法，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 8B - The method of Aspect 1B, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面9B-一種被配置為處理視訊資料的裝置，所述裝置包括：被配置為儲存圖片的記憶體；以及在電路中實施並且與所述記憶體相通訊的一個或多個處理器，所述一個或多個處理器被配置為：接收所述圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 9B - An apparatus configured to process video data, the apparatus comprising: a memory configured to store pictures; and one or more processors implemented in circuitry and in communication with the memory, the The one or more processors are configured to: receive the picture; and encode a picture orientation message including a transform type syntax element, wherein the transform type syntax element indicates a transformation from among the plurality of transforms to be applied to the picture. transformation of the picture.

方面10B-根據方面9B所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。Aspect 10B - The apparatus of aspect 9B, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of rotation and mirror transformations.

方面11B-根據方面9B所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 11B - The apparatus of Aspect 9B, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面12B-根據方面9B所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 12B - The apparatus of Aspect 9B, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面13B-根據方面9B所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 13B - The apparatus of Aspect 9B, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the The persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面14B-根據方面9B所述的裝置，其中，所述裝置被配置為對所述圖片方位訊息進行解碼，並且其中，所述一個或多個處理器還被配置為：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 14B - The apparatus of Aspect 9B, wherein the apparatus is configured to decode the picture orientation information, and wherein the one or more processors are further configured to: elements to apply the transformation to the picture to form a transformed picture; and display the transformed picture.

方面15B-根據方面9B所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 15B - The device of Aspect 9B, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面16B-根據方面9B所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 16B - The device of Aspect 9B, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面17B-一種被配置為處理視訊資料的裝置，所述裝置包括：用於接收圖片的部件；以及用於對包括變換類型語法元素的圖片方位訊息進行寫碼的部件，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 17B - An apparatus configured to process video data, the apparatus comprising: means for receiving a picture; and means for encoding picture orientation information comprising a transform type syntax element, wherein the transform type A syntax element indicates a transform from among a plurality of transforms to apply to the picture.

方面18B-根據方面17B所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。Aspect 18B - The apparatus of aspect 17B, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of the rotation transformation and the mirror transformation.

方面19B-根據方面17B所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 19B - The apparatus of Aspect 17B, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面20B-根據方面17B所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 20B - The apparatus of aspect 17B, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面21B-根據方面17B所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 21B - The apparatus of aspect 17B, wherein the picture orientation message further comprises a cancel flag and a persistence flag, wherein the cancel flag indicates whether a previous persistence flag is canceled, and wherein the The persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面22B-根據方面17B所述的裝置，其中，所述用於寫碼的部件包括用於解碼的部件，並且其中，所述裝置還包括：用於根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片的部件；以及用於顯示所述經變換的圖片的部件。Aspect 22B - The apparatus of aspect 17B, wherein said means for writing comprises means for decoding, and wherein said apparatus further comprises: for encoding said means for applying the transformation to a picture to form a transformed picture; and means for displaying the transformed picture.

方面23B-根據方面17B所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 23B - The device of aspect 17B, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面24B-根據方面17B所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 24B - The device of Aspect 17B, wherein the picture orientation information comprises a picture orientation open bitstream unit (OBU).

方面25B-一種儲存指令的非暫時性電腦可讀儲存媒體，所述指令當被執行時使得被配置為處理視訊資料的設備的一個或多個處理器進行以下操作：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 25B - A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to process video data to: receive a picture; The picture orientation information is encoded with a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture.

方面26B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。Aspect 26B - The non-transitory computer readable storage medium of aspect 25B, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a rotation transformation and a mirror transformation The combination.

方面27B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 27B - The non-transitory computer readable storage medium of aspect 25B, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; A third transformation comprising said 180 degree counterclockwise transformation followed by said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said a 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面28B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 28B - The non-transitory computer-readable storage medium of aspect 25B, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面29B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 29B - The non-transitory computer readable storage medium of Aspect 25B, wherein the picture orientation information further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面30B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述設備被配置為對所述圖片方位訊息進行解碼，並且其中，所述指令還使得所述一個或多個處理器進行以下操作：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 30B - The non-transitory computer readable storage medium of aspect 25B, wherein the device is configured to decode the picture orientation information, and wherein the instructions further cause the one or more processes The processor applies the transform to the picture according to the transform-type syntax element to form a transformed picture; and displays the transformed picture.

方面31B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 31B - The non-transitory computer readable storage medium of aspect 25B, wherein the picture orientation information comprises a picture orientation supplemental enhancement information (SEI) message.

方面32B-根據方面25B所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 32B - The non-transitory computer readable storage medium of Aspect 25B, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面1C-一種處理視訊資料的方法，所述方法包括：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 1C - A method of processing video data, the method comprising: receiving a picture; and encoding picture orientation information including a transform type syntax element, wherein the transform type syntax element indicates a request from among a plurality of transforms The transform to apply to the image.

方面2C-根據方面1C所述的方法，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。Aspect 2C - The method of aspect 1C, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of the rotation transformation and the mirror transformation.

方面3C-根據方面1C所述的方法，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 3C - The method of Aspect 1C, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面4C-根據方面1C-3C中任一項所述的方法，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 4C - The method of any of aspects 1C-3C, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面5C-根據方面1C-4C中任一項所述的方法，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 5C - The method of any of aspects 1C-4C, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether a previous persistence flag is canceled and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面6C-根據方面1C-5C中任一項所述的方法，其中，寫碼包括解碼，並且其中，所述方法還包括：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 6C - The method of any of aspects 1C-5C, wherein encoding comprises decoding, and wherein the method further comprises applying the transform to the picture according to the transform type syntax element, to form a transformed picture; and displaying the transformed picture.

方面7C-根據方面1C-6C中任一項所述的方法，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 7C - The method of any one of aspects 1C-6C, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面8C-根據方面1C-6C中任一項所述的方法，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 8C - The method of any one of aspects 1C-6C, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面9C-一種被配置為處理視訊資料的裝置，所述裝置包括：被配置為儲存圖片的記憶體；以及在電路中實施並且與所述記憶體相通訊的一個或多個處理器，所述一個或多個處理器被配置為：接收所述圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 9C - An apparatus configured to process video data, the apparatus comprising: a memory configured to store pictures; and one or more processors implemented in circuitry and in communication with the memory, the The one or more processors are configured to: receive the picture; and encode a picture orientation message including a transform type syntax element, wherein the transform type syntax element indicates a transformation from among the plurality of transforms to be applied to the picture. transformation of the picture.

方面10C-根據方面9C所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。Aspect 10C - The apparatus of aspect 9C, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of rotation and mirror transformations.

方面11C-根據方面9C所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 11C - The apparatus of Aspect 9C, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面12C-根據方面9C-11C中任一項所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 12C - The apparatus of any one of aspects 9C-11C, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面13C-根據方面9C-12C中任一項所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 13C - The apparatus of any of aspects 9C-12C, wherein the picture orientation message further comprises a cancel flag and a persistence flag, wherein the cancel flag indicates whether a previous persistence flag is canceled and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面14C-根據方面9C-13C中任一項所述的裝置，其中，所述裝置被配置為對所述圖片方位訊息進行解碼，並且其中，所述一個或多個處理器還被配置為：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 14C - The apparatus of any of aspects 9C-13C, wherein the apparatus is configured to decode the picture orientation information, and wherein the one or more processors are further configured to: applying the transform to the picture according to the transform-type syntax element to form a transformed picture; and displaying the transformed picture.

方面15C-根據方面9C-14C中任一項所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 15C - The device of any one of aspects 9C-14C, wherein the picture orientation information comprises a picture orientation supplemental enhancement information (SEI) message.

方面16C-根據方面9C-14C中任一項所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 16C - The apparatus of any one of aspects 9C-14C, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面17C-一種被配置為處理視訊資料的裝置，所述裝置包括：用於接收圖片的部件；以及用於對包括變換類型語法元素的圖片方位訊息進行寫碼的部件，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 17C - An apparatus configured to process video data, the apparatus comprising: means for receiving a picture; and means for encoding picture orientation information comprising a transform type syntax element, wherein the transform type A syntax element indicates a transform from among a plurality of transforms to apply to the picture.

方面18C-根據方面17C所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。Aspect 18C - The apparatus of Aspect 17C, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a combination of the rotation transformation and the mirror transformation.

方面19C-根據方面17C所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 19C - The apparatus of Aspect 17C, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising the following is said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面20C-根據方面17C-19C中任一項所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 20C - The apparatus of any one of aspects 17C-19C, wherein the transform type syntax element further comprises a value indicating that no transform will be applied.

方面21C-根據方面17C-20C中任一項所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 21C - The apparatus of any of aspects 17C-20C, wherein the picture orientation message further comprises a cancel flag and a persistence flag, wherein the cancel flag indicates whether a previous persistence flag is canceled and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面22C-根據方面17C-21C中任一項所述的裝置，其中，所述用於寫碼的部件包括用於解碼的部件，並且其中，所述裝置還包括：用於根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片的部件；以及用於顯示所述經變換的圖片的部件。Aspect 22C - The apparatus of any of aspects 17C-21C, wherein said means for writing comprises means for decoding, and wherein said apparatus further comprises: means for applying the transformation to the picture, syntax elements to form a transformed picture; and means for displaying the transformed picture.

方面23C-根據方面17C-22C中任一項所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 23C - The device of any one of aspects 17C-22C, wherein the picture orientation information comprises picture orientation supplemental enhancement information (SEI) information.

方面24C-根據方面17C-22C中任一項所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 24C - The apparatus of any one of aspects 17C-22C, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU).

方面25C-一種儲存指令的非暫時性電腦可讀儲存媒體，所述指令當被執行時使得被配置為處理視訊資料的設備的一個或多個處理器進行以下操作：接收圖片；以及對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。Aspect 25C - A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to process video data to: receive a picture; The picture orientation information is encoded with a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture.

方面26C-根據方面25C所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。Aspect 26C - The non-transitory computer readable storage medium of aspect 25C, wherein the plurality of transformations comprises two or more of: a rotation transformation, a mirror transformation, or a rotation transformation and a mirror transformation The combination.

方面27C-根據方面25C所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。Aspect 27C - The non-transitory computer readable storage medium of aspect 25C, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; A third transformation comprising said 180 degree counterclockwise transformation followed by said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said a 90 degree counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation.

方面28C-根據方面25C-27C中任一項所述的非暫時性電腦可讀儲存媒體，其中，所述變換類型語法元素還包括指示將不應用變換的值。Aspect 28C - The non-transitory computer-readable storage medium of any of aspects 25C-27C, wherein the transform type syntax element further includes a value indicating that no transform will be applied.

方面29C-根據方面25C-28C中任一項所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。Aspect 29C - The non-transitory computer readable storage medium of any one of aspects 25C-28C, wherein the picture orientation information further comprises a cancellation flag and a persistence flag, wherein the cancellation flag indicates whether to cancel the previous persistence flag, and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures.

方面30C-根據方面25C-29C中任一項所述的非暫時性電腦可讀儲存媒體，其中，所述設備被配置為對所述圖片方位訊息進行解碼，並且其中，所述指令還使得所述一個或多個處理器進行以下操作：根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及顯示所述經變換的圖片。Aspect 30C - The non-transitory computer readable storage medium of any one of aspects 25C-29C, wherein the device is configured to decode the picture orientation information, and wherein the instructions further cause the The one or more processors: apply the transform to the picture according to the transform-type syntax element to form a transformed picture; and display the transformed picture.

方面31C-根據方面25C-30C中任一項所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。Aspect 31C - The non-transitory computer readable storage medium of any one of aspects 25C-30C, wherein the picture orientation information comprises a picture orientation supplemental enhancement information (SEI) message.

方面32C-根據方面25C-30C中任一項所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。Aspect 32C - The non-transitory computer readable storage medium of any one of aspects 25C-30C, wherein the picture orientation information comprises a picture orientation open bitstream unit (OBU).

要認識到的是，根據示例，本文描述的技術中的任何技術的某些動作或事件可以以不同的順序執行，可以被添加、被合併或被完全省略(例如，並非所有描述的動作或事件對於技術的實施都是必要的)。此外，在某些示例中，動作或事件可以例如通過多線程處理、中斷處理或多個處理器併發地而不是順序地執行。It is to be appreciated that, depending on the example, certain acts or events of any of the techniques described herein may be performed in a different order, added to, combined, or omitted entirely (e.g., not all described acts or events necessary for the implementation of the technology). Furthermore, in some examples, actions or events may be performed concurrently rather than sequentially, eg, through multithreading, interrupt handling, or multiple processors.

在一個或多個示例中，所描述的功能可以用硬體、軟體、韌體或其任何組合來實施。如果用軟體來實施，則所述功能可以作為一個或多個指令或代碼被儲存在電腦可讀媒體上或者通過電腦可讀媒體進行發送並且由基於硬體的處理單元執行。電腦可讀媒體可以包括電腦可讀儲存媒體(其對應於諸如資料儲存媒體的有形媒體)或者通訊媒體(其包括例如根據通訊協定來促進電腦程式從一個地方傳送到另一個地方的任何媒體)。以這種方式，電腦可讀媒體通常可以對應於(1)非暫時性的有形電腦可讀儲存媒體、或者(2)諸如訊號或載波的通訊媒體。資料儲存媒體可以是能夠由一個或多個電腦或者一個或多個處理器存取以取回用於實施在本揭露內容中描述的技術的指令、代碼和/或資料結構的任何可用的媒體。電腦程式產品可以包括電腦可讀媒體。In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over, as one or more instructions or code, a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media (which correspond to tangible media such as data storage media) or communication media (which includes any medium that facilitates transfer of a computer program from one place to another, eg, according to a communication protocol). In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include computer readable media.

舉例來說而非進行限制，這樣的電腦可讀儲存媒體可以包括RAM、ROM、EEPROM、CD-ROM或其它光碟儲存、磁碟儲存或其它磁儲存設備、快閃記憶體、或者能夠用於以指令或資料結構的形式儲存期望的程式代碼以及能夠由電腦存取的任何其它媒體。此外，任何連接被適當地稱為電腦可讀媒體。例如，如果使用同軸電纜、光纖光纜、雙絞線、數位訂戶線(DSL)或者無線技術(諸如紅外線、無線電和微波)來從網站、伺服器或其它遠程源發送指令，則同軸電纜、光纖光纜、雙絞線、DSL或者無線技術(諸如紅外線、無線電和微波)被包括在媒體的定義中。然而，應當理解的是，電腦可讀儲存媒體和資料儲存媒體不包括連接、載波、訊號或其它暫時性媒體，而是替代地針對非暫時性的有形儲存媒體。如本文所使用的，磁碟和光碟包括壓縮光碟(compact disc, CD)、激光光碟、光碟、數位多功能光碟(digital versatile disc, DVD)、軟碟和藍光光碟，其中，磁碟通常磁性地複製資料，而光碟利用激光來光學地複製資料。上述各項的組合還應當被包括在電腦可讀媒體的範圍之內。By way of example and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or can be used in Instructions or data structures store desired program code and any other medium that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave) are used to send instructions from a website, server, or other remote source, then coaxial cable, fiber optic cable , twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disc, and Blu-ray disc, where discs are usually magnetically copy data, while optical discs use laser light to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.

指令可以由一個或多個處理器(諸如一個或多個DSP、通用微處理器、ASIC、FPGA、或其它等效的整合或離散邏輯電路)執行。因此，如本文所使用的術語“處理器”和“處理電路”可以指代前述結構中的任何一者或者適於實施本文描述的技術的任何其它結構。另外，在一些方面中，本文描述的功能可以在被配置用於編碼和解碼的專用硬體和/或軟體模組內提供，或者被併入經組合的編解碼器中。此外，所述技術可以在一個或多個電路或邏輯元件中完全實施。Instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Accordingly, the terms "processor" and "processing circuitry," as used herein, may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

本揭露內容的技術可以在多種多樣的設備或裝置(包括無線手持機、積體電路(integrated circuit, IC)或一組IC(例如，晶片組))中實施。在本揭露內容中描述了各種組件、模組或單元以強調被配置為執行所揭露的技術的設備的功能方面，但是不一定要求通過不同的硬體單元來實施。確切而言，如上所述，各種單元可以被組合在編解碼器硬體單元中，或者由互操作的硬體單元的集合(包括如上文所描述的一個或多個處理器)結合適當的軟體和/或韌體來提供。The techniques of this disclosure may be implemented in a wide variety of devices or devices, including wireless handsets, integrated circuits (ICs), or groups of ICs (eg, chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, or a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and/or firmware to provide.

已經描述了各個示例。這些示例和其它示例在所附的請求項的範圍內。Various examples have been described. These examples and others are within the scope of the appended claims.

100:系統 102:源設備 104:視訊源 106:記憶體 108:輸出介面 200:視訊編碼器 110:電腦可讀媒體 112:儲存設備 114:檔案伺服器 116:目的地設備 118:顯示設備 120:記憶體 122:輸入介面 300:視訊解碼器 150:圖片 152:圖片 160:原始圖片 162:圖片 164:圖片 166:圖片 168:圖片 170:圖片 172:圖片 174:圖片 400:步驟 402:步驟 404:步驟 410:步驟 412:步驟 414:步驟 416:步驟 500:步驟 502:步驟 504:步驟 510:步驟 512:步驟 514:步驟 516:步驟 202:模式選擇單元 222:運動估計單元 224:運動補償單元 226:幀內預測單元 204:殘差生成單元 206:變換處理單元 208:量化單元 210:逆量化單元 212:逆變換處理單元 214:重構單元 216:濾波器單元 218:解碼圖片緩衝器 220:熵編碼單元 230:視訊資料記憶體 302:熵解碼單元 304:預測處理單元 316:運動補償單元 318:幀內預測單元 306:逆量化單元 308:逆變換處理單元 310:重構單元 312:濾波器單元 314:解碼圖片緩衝器 320:寫碼圖片緩衝器記憶體 350:步驟 352:步驟 354:步驟 356:步驟 358:步驟 360:步驟 370:步驟 372:步驟 374:步驟 376:步驟 378:步驟 380:步驟 100: system 102: Source device 104: Video source 106: Memory 108: output interface 200: video encoder 110: Computer-readable media 112: storage equipment 114:File server 116:Destination device 118: Display device 120: memory 122: input interface 300: Video decoder 150: picture 152: picture 160:Original picture 162: picture 164: picture 166: picture 168: picture 170: picture 172: picture 174: picture 400: step 402: step 404: step 410: Step 412: Step 414:step 416: step 500: step 502: Step 504: step 510: step 512: Step 514: step 516: step 202: mode selection unit 222: Motion Estimation Unit 224: Motion Compensation Unit 226: Intra prediction unit 204: residual generation unit 206: transformation processing unit 208: quantization unit 210: inverse quantization unit 212: inverse transform processing unit 214: Refactoring unit 216: filter unit 218: Decode picture buffer 220: entropy coding unit 230: Video data memory 302: entropy decoding unit 304: prediction processing unit 316: Motion Compensation Unit 318: Intra prediction unit 306: inverse quantization unit 308: inverse transform processing unit 310: Refactoring unit 312: filter unit 314: Decode picture buffer 320: Write code picture buffer memory 350: step 352: Step 354: step 356: step 358: Step 360: steps 370: step 372: step 374: step 376: step 378:step 380: Step

圖1是示出可以執行本揭露內容的技術的示例視訊編碼和解碼系統的方塊圖。1 is a block diagram illustrating an example video encoding and decoding system that may implement techniques of this disclosure.

圖2是示出圖片的示例旋轉的概念圖。FIG. 2 is a conceptual diagram illustrating an example rotation of a picture.

圖3是示出示例變換類型的概念圖。FIG. 3 is a conceptual diagram illustrating example transformation types.

圖4是示出用於對圖片方位補充增強資訊訊息進行寫碼的示例過程的流程圖。4 is a flowchart illustrating an example process for encoding picture orientation supplemental augmentation information messages.

圖5是示出用於對品質度量補充增強資訊訊息進行寫碼的示例過程的流程圖。5 is a flowchart illustrating an example process for coding a quality metric supplemental augmentation information message.

圖6是示出可以執行本揭露內容的技術的示例視訊編碼器的方塊圖。6 is a block diagram illustrating an example video encoder that may perform techniques of this disclosure.

圖7是示出可以執行本揭露內容的技術的示例視訊解碼器的方塊圖。7 is a block diagram illustrating an example video decoder that may perform techniques of this disclosure.

圖8是示出根據本揭露內容的技術的用於對當前區塊進行編碼的示例方法的流程圖。8 is a flowchart illustrating an example method for encoding a current block in accordance with the techniques of this disclosure.

圖9是示出根據本揭露內容的技術的用於對當前區塊進行解碼的示例方法的流程圖。9 is a flowchart illustrating an example method for decoding a current block in accordance with the techniques of this disclosure.

102:源設備 102: Source device

116:目的地設備 116:Destination device

400:步驟 400: step

402:步驟 402: step

404:步驟 404: step

410:步驟 410: Step

412:步驟 412: Step

414:步驟 414:step

416:步驟 416: step

Claims (32)

一種處理視訊資料的方法，所述方法包括： 接收圖片；以及 對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。 A method of processing video data, the method comprising: receive pictures; and A picture orientation message is encoded that includes a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. 如請求項1所述的方法，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。The method of claim 1, wherein the plurality of transformations includes two or more of the following: a rotation transformation, a mirror transformation, or a combination of the rotation transformation and the mirror transformation. 如請求項1所述的方法，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。The method as claimed in claim 1, wherein said plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising followed by the said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation , comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation. 如請求項1所述的方法，其中，所述變換類型語法元素還包括指示將不應用變換的值。The method of claim 1, wherein the transform type syntax element further includes a value indicating that no transform will be applied. 如請求項1所述的方法，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。The method of claim 1, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the persistence A sex flag indicates whether the picture orientation information applies to subsequent pictures. 如請求項1所述的方法，其中，寫碼包括解碼，並且其中，所述方法還包括： 根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及 顯示所述經變換的圖片。 The method according to claim 1, wherein writing codes includes decoding, and wherein the method further includes: applying the transform to the picture according to the transform-type syntax element to form a transformed picture; and The transformed picture is displayed. 如請求項1所述的方法，其中，所述圖片方位訊息包括圖片方位補充增強資訊(supplemental enhancement information, SEI)訊息。The method according to claim 1, wherein the picture orientation information includes picture orientation supplemental enhancement information (supplemental enhancement information, SEI) information. 如請求項1所述的方法，其中，所述圖片方位訊息包括圖片方位開放位元流單元(open bitstream unit, OBU)。The method according to claim 1, wherein the picture orientation information includes a picture orientation open bitstream unit (OBU). 一種被配置為處理視訊資料的裝置，所述裝置包括： 被配置為儲存圖片的記憶體；以及 在電路中實施並且與所述記憶體相通訊的一個或多個處理器，所述一個或多個處理器被配置為： 接收所述圖片；以及 對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。 An apparatus configured to process video data, the apparatus comprising: memory configured to store pictures; and one or more processors embodied in circuitry and in communication with said memory, said one or more processors being configured to: receiving said picture; and A picture orientation message is encoded that includes a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. 如請求項9所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。The apparatus according to claim 9, wherein the plurality of transformations include two or more of the following: rotation transformation, mirror transformation, or a combination of rotation transformation and mirror transformation. 如請求項9所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。The apparatus of claim 9, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180-degree counterclockwise rotation transformation; a third transformation comprising followed by the said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation , comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation. 如請求項9所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。The apparatus of claim 9, wherein the transform type syntax element further includes a value indicating that no transform will be applied. 如請求項9所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。The device according to claim 9, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the persistence A sex flag indicates whether the picture orientation information applies to subsequent pictures. 如請求項9所述的裝置，其中，所述裝置被配置為對所述圖片方位訊息進行解碼，並且其中，所述一個或多個處理器還被配置為： 根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及 顯示所述經變換的圖片。 The apparatus of claim 9, wherein the apparatus is configured to decode the picture orientation information, and wherein the one or more processors are further configured to: applying the transform to the picture according to the transform-type syntax element to form a transformed picture; and The transformed picture is displayed. 如請求項9所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。The device according to claim 9, wherein the picture orientation information includes picture orientation Supplemental Enhancement Information (SEI) information. 如請求項9所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。The device according to claim 9, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU). 一種被配置為處理視訊資料的裝置，所述裝置包括： 用於接收圖片的部件；以及 用於對包括變換類型語法元素的圖片方位訊息進行寫碼的部件，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。 An apparatus configured to process video data, the apparatus comprising: a component for receiving pictures; and Means for encoding picture orientation information including a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. 如請求項17所述的裝置，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者所述旋轉變換和所述鏡像變換的組合。The apparatus according to claim 17, wherein the plurality of transformations include two or more of the following: rotation transformation, mirror transformation, or a combination of the rotation transformation and the mirror transformation. 如請求項17所述的裝置，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。The apparatus of claim 17, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180 degree counterclockwise rotation transformation; a third transformation comprising followed by the said 180 degree counterclockwise transformation of said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree counterclockwise transformation; a sixth transformation , comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation. 如請求項17所述的裝置，其中，所述變換類型語法元素還包括指示將不應用變換的值。The apparatus of claim 17, wherein the transform type syntax element further includes a value indicating that no transform will be applied. 如請求項17所述的裝置，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。The apparatus according to claim 17, wherein the picture orientation message further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag, and wherein the persistence A sex flag indicates whether the picture orientation information applies to subsequent pictures. 如請求項17所述的裝置，其中，所述用於寫碼的部件包括用於解碼的部件，並且其中，所述裝置還包括： 用於根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片的部件；以及 用於顯示所述經變換的圖片的部件。 The device according to claim 17, wherein the means for writing codes includes means for decoding, and wherein the device further comprises: means for applying the transform to the picture according to the transform-type syntax element to form a transformed picture; and The component used to display the transformed picture. 如請求項17所述的裝置，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。The device according to claim 17, wherein the picture orientation information comprises a picture orientation Supplemental Enhancement Information (SEI) message. 如請求項17所述的裝置，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。The device of claim 17, wherein the picture orientation information comprises a picture orientation Open Bitstream Unit (OBU). 一種儲存指令的非暫時性電腦可讀儲存媒體，所述指令當被執行時使得被配置為處理視訊資料的設備的一個或多個處理器進行以下操作： 接收圖片；以及 對包括變換類型語法元素的圖片方位訊息進行寫碼，其中，所述變換類型語法元素指示來自在多個變換當中的要應用於所述圖片的變換。 A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to process video data to: receive pictures; and A picture orientation message is encoded that includes a transform type syntax element indicating a transform from among a plurality of transforms to be applied to the picture. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括以下各項中的兩項或更多項：旋轉變換、鏡像變換、或者旋轉變換和鏡像變換的組合。The non-transitory computer-readable storage medium of claim 25, wherein the plurality of transformations includes two or more of the following: rotation transformations, mirror transformations, or a combination of rotation transformations and mirror transformations . 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述多個變換包括：第一變換，其包括水平鏡像變換；第二變換，其包括180度逆時針旋轉變換；第三變換，其包括隨後是所述水平鏡像變換的所述180度逆時針變換；第四變換，其包括隨後是所述水平鏡像變換的90度逆時針變換；第五變換，其包括所述90度逆時針變換；第六變換，其包括隨後是所述水平鏡像變換的270度逆時針變換；以及第七變換，其包括所述270度逆時針變換。The non-transitory computer-readable storage medium of claim 25, wherein the plurality of transformations comprises: a first transformation comprising a horizontal mirror transformation; a second transformation comprising a 180-degree counterclockwise rotation transformation; a third a transformation comprising said 180 degree counterclockwise transformation followed by said horizontal mirror transformation; a fourth transformation comprising said 90 degree counterclockwise transformation followed by said horizontal mirror transformation; a fifth transformation comprising said 90 degree a counterclockwise transformation; a sixth transformation comprising a 270 degree counterclockwise transformation followed by said horizontal mirror transformation; and a seventh transformation comprising said 270 degree counterclockwise transformation. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述變換類型語法元素還包括指示將不應用變換的值。The non-transitory computer readable storage medium of claim 25, wherein the transform type syntax element further includes a value indicating that no transform will be applied. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息還包括取消旗標和持續性旗標，其中，所述取消旗標指示是否取消先前的持續性旗標，並且其中，所述持續性旗標指示所述圖片方位訊息是否適用於後續圖片。The non-transitory computer-readable storage medium according to claim 25, wherein the picture orientation information further includes a cancel flag and a persistence flag, wherein the cancel flag indicates whether to cancel a previous persistence flag , and wherein the persistence flag indicates whether the picture orientation information applies to subsequent pictures. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述設備被配置為對所述圖片方位訊息進行解碼，並且其中，所述指令還使得所述一個或多個處理器進行以下操作： 根據所述變換類型語法元素來對所述圖片應用所述變換，以形成經變換的圖片；以及 顯示所述經變換的圖片。 The non-transitory computer readable storage medium of claim 25, wherein the device is configured to decode the picture orientation information, and wherein the instructions further cause the one or more processors to perform Do the following: applying the transform to the picture according to the transform-type syntax element to form a transformed picture; and The transformed picture is displayed. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位補充增強資訊(SEI)訊息。The non-transitory computer-readable storage medium of claim 25, wherein the picture orientation information includes a picture orientation Supplemental Enhancement Information (SEI) message. 如請求項25所述的非暫時性電腦可讀儲存媒體，其中，所述圖片方位訊息包括圖片方位開放位元流單元(OBU)。The non-transitory computer-readable storage medium of claim 25, wherein the picture orientation information includes a picture orientation Open Bitstream Unit (OBU).
TW111108835A 2021-04-02 2022-03-10 Picture orientation and quality metrics supplemental enhancement information message for video coding TW202243482A (en)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US202163170267P 2021-04-02 2021-04-02
US63/170,267 2021-04-02
US202163214378P 2021-06-24 2021-06-24
US63/214,378 2021-06-24
US17/653,947 US20220321918A1 (en) 2021-04-02 2022-03-08 Picture orientation and quality metrics supplemental enhancement information message for video coding
US17/653,947 2022-03-08

Publications (1)

Publication Number Publication Date
TW202243482A true TW202243482A (en) 2022-11-01

Family

ID=80952318

Family Applications (1)

Application Number Title Priority Date Filing Date
TW111108835A TW202243482A (en) 2021-04-02 2022-03-10 Picture orientation and quality metrics supplemental enhancement information message for video coding

Country Status (6)

Country Link
EP (1) EP4315864A1 (en)
JP (1) JP2024512601A (en)
KR (1) KR20230159428A (en)
BR (1) BR112023018967A2 (en)
TW (1) TW202243482A (en)
WO (1) WO2022213005A1 (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103416002B (en) * 2011-03-10 2015-04-01 维德约股份有限公司 Render-orientation information in video bitstream
US9438818B2 (en) * 2012-06-20 2016-09-06 Qualcomm Incorporated Device and method for multimedia communications with picture orientation information
US11140373B2 (en) * 2018-04-05 2021-10-05 Lg Electronics Inc. Method for transmitting 360-degree video, method for receiving 360-degree video, apparatus for transmitting 360-degree video, and apparatus for receiving 360-degree video

Also Published As

Publication number Publication date
EP4315864A1 (en) 2024-02-07
KR20230159428A (en) 2023-11-21
JP2024512601A (en) 2024-03-19
WO2022213005A1 (en) 2022-10-06
BR112023018967A2 (en) 2023-10-17

Similar Documents

Publication Publication Date Title
CN115336279A (en) General constraint information syntax in video coding
CN114258675A (en) Cross-component adaptive loop filtering for video coding
CN114846794A (en) Reference sub-picture scaling ratio for sub-pictures in video coding
CN114424566A (en) Quantization parameter signaling for joint chroma residual mode in video coding
EP3997875A1 (en) Deriving coding system operational configuration
WO2019160795A1 (en) Intra prediction for 360-degree video
CN116158080A (en) Multiple adaptive loop filter sets
CN114982233A (en) Signaling scaling matrices in video coding
CN114830667A (en) Reference picture scaling for reference picture resampling in video coding and decoding
CN114868387A (en) Chroma transform skipping and joint chroma coding enablement for blocks in video coding
TW202143711A (en) High level syntax for video with mixed nal unit types
CN112913245A (en) Intra-block copy prediction restriction in video coding
CN115244934A (en) Video coding and decoding syntax
CN113994706A (en) Chroma delta quantization parameters in video coding
TW202205865A (en) Deblocking filter parameter signaling
CN114930821A (en) Flexible signaling of QP offset for adaptive color transform in video coding
CN114868400A (en) Equation-based Rice parameter derivation for conventional transform coefficients in video coding
US10779006B2 (en) Signaling 360-degree video information
US11895336B2 (en) Picture orientation and quality metrics supplemental enhancement information message for video coding
CN115516866A (en) Block partitioning for picture and video coding
CN114402599A (en) Signaling of sub-pictures in high level syntax for video coding
CN113994675A (en) Maximum allowed block size for BDPCM mode
TW202243482A (en) Picture orientation and quality metrics supplemental enhancement information message for video coding
TW202241134A (en) Picture orientation and quality metrics supplemental enhancement information message for video coding
CN116848845A (en) Picture orientation and quality metric supplemental enhancement information message for video coding