CN112449233A - 解码传入的音频/视频***的方法 - Google Patents

解码传入的音频/视频***的方法 Download PDF

Info

Publication number
CN112449233A
CN112449233A CN202010914890.2A CN202010914890A CN112449233A CN 112449233 A CN112449233 A CN 112449233A CN 202010914890 A CN202010914890 A CN 202010914890A CN 112449233 A CN112449233 A CN 112449233A
Authority
CN
China
Prior art keywords
audio
video
delay
decoding
condition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010914890.2A
Other languages
English (en)
Inventor
S·德尔普拉斯
F·索迪
J·伯格
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sagemcom Broadband SAS
Original Assignee
Sagemcom Broadband SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sagemcom Broadband SAS filed Critical Sagemcom Broadband SAS
Publication of CN112449233A publication Critical patent/CN112449233A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4343—Extraction or processing of packetized elementary streams [PES]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/04—Time compression or expansion
    • G10L21/055—Time compression or expansion for synchronising with other signals, e.g. video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/36—Monitoring, i.e. supervising the progress of recording or reproducing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/765—Media network packet handling intermediate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/65—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience
    • H04N19/68—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using error resilience involving the insertion of resynchronisation markers into the bitstream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23608—Remultiplexing multiplex streams, e.g. involving modifying time stamps or remapping the packet identifiers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4344—Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4347—Demultiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438—Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H04N21/4382—Demodulation or channel decoding, e.g. QPSK demodulation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising
    • H04N5/06—Generation of synchronising signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/611—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for multicast or broadcast

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Television Receiver Circuits (AREA)

Abstract

一种解码由连接到音频回放装备(12)和视频回放装备(13)的解码器盒(11)接收到的传入的音频/视频流的方法，所述方法包括以下步骤：·估算音频延时和视频延时；·如果音频(或相应地视频)延时大于视频(或相应地音频)延时，则测试第一条件以确保音频(或相应地视频)样本与基准时钟信号之间的时间差(Δt_a，Δt_v)是足够的，并且如果满足第一条件，则在解码音频(或相应地视频)时采取操作以降低音频(或相应地视频)样本的演示时间戳以提前回放音频(或相应地视频)样本，由此至少部分地补偿音频(或相应地视频)延时和视频(或相应地音频)延时之间的延时差异。

Description

解码传入的音频/视频***的方法
技术领域
本发明涉及解码器盒和解码传入的音频/视频流的领域。
背景技术
遵从ISO/IEC 13818 1标准来操作的解码器盒(通常称为“机顶盒”)接收传入的音频/视频流，解码该传入的音频/视频流，并在回放它之前对它进行同步。
传入的音频/视频流可通过电缆、通过卫星、或通过网际协议(IP)链路(例如，DASH-TS或HLS-TS链路)被传送到解码器盒。
传入的音频/视频流包括复用的音频样本和视频样本。诸样本由相应的演示时间戳(PTS)来加时间戳。演示时间戳定义样本要被呈现(对音频而言是播放，对视频而言是显示)的时间。
传入的音频/视频流运送称为节目时钟基准(PCR)的基准时钟信号。解码器盒利用基准时钟信号来对其内部时钟进行伺服控制(称为***时间时钟(STC))并正确地显示诸样本。
视频随后一般被显示在电视机上。
对于音频，它可由电视机、或者由音频/视频放大器(例如，连接到解码器盒的HDMI端口或S/PDIF端口)、或者由经由HDMI端口或S/PDIF端口或者通过Wi-Fi或蓝牙类型的无线传输直接连接到解码器盒的某一其他音频回放装备来播放。作为示例，音频回放装备可以是音箱或智能扬声器。
此类装备引入尤其得自各装备件的内部处理时间的延时。这一延时可以按各种方式来估算。作为示例，延时可由装备自身通过经由HDMI端口传送经扩展显示标识数据(EDID)直接提供。同样作为示例，延时可通过测量传送和接收声音信号之间的偏移来估算。
如果音频和视频不是被同一件装备来呈现，则音频路径的音频延时和视频路径的视频延时需要相等以避免引入声音和图像之间的任何用户可感知的偏移。作为示例，此类偏移可引起唇同步的问题。
以上是参考附图1来解说的，其示出了连接到扬声器2和电视机3的解码器盒1。
解码器盒1包括音频解码器模块4和音频捕获模块5。音频路径包括音频解码器模块4、音频捕获模块5和扬声器2。
解码器盒1还包括视频解码器模块6。视频路径包括视频解码器模块6和电视机3。
解码器盒1还包括用于将音频样本和视频样本相对于基准时钟信号进行同步的同步机制7。
在图1的示例中，假定扬声器2在音频路径中引入90毫秒(ms)的延时。音频捕获模块5在音频路径中引入80ms的延时。总音频延时因而是170ms。
电视机3在视频路径中引入70ms的延时。总视频延时因而是70ms。
音频延时因而比视频延时大100ms：因此在视频路径中需要延迟。
参考附图2，扬声器2同样在音频路径中引入90ms的延时。音频捕获模块5在音频路径中引入80ms的延时。总音频延时因而是170ms。
在此，电视机3在视频路径中引入190ms的延时。总视频延时因而是190ms。
视频延时因而比音频延时大20ms：因此在音频路径中需要延迟。
在这两种情形中，这些延迟由同步机制7施加使声音或图像的解码延迟的“解码延迟”而被引入在诸路径中。解码延迟是正数且与音频路径或视频路径中引入的“偏移”相对应。
在图1的示例中，视频解码延迟等于100ms，而在图2的示例中，音频解码延迟等于20ms。
在音频路径或视频路径中引入解码延迟具有对用户体验的负面影响。
具体而言，解码延迟被施加到传入的音频/视频流上的每一用户动作。因而，作为示例，解码延迟增加了“频道跳转”(即，更换电视频道所需的时间)所花的时间。取决于所使用的音频播放装备，对于在最佳情形中800ms量级的总频道跳转时间，频道跳转时间因而通常增加100ms到200ms。解码延迟的效果因而远不是可忽略的。
发明目的
本发明的目标是改进解码器盒的用户体验，并且按简单且便宜的方式这样做。
发明内容
为了实现这一目标，提供了一种用于解码由连接到音频回放装备和视频回放装备的解码器盒接收到的传入的音频/视频流的解码方法，该传入的音频/视频流运送基准时钟信号并包括包含相应演示时间戳的音频样本和视频样本，该解码方法包括以下步骤：
·估算由音频路径引入的音频延时和由视频路径引入的视频延时；
·如果音频(或相应地视频)延时大于视频(或相应地音频)延时，则测试第一条件以确保音频(或相应地视频)样本与基准时钟信号之间的时间差足以使得能够正确地解码音频(或相应地视频)样本，并且如果满足第一条件，则在解码音频(或相应地视频)时采取操作以降低音频(或相应地视频)样本的演示时间戳以提前回放音频(或相应地视频)样本，由此至少部分地补偿音频(或相应地视频)延时和视频(或相应地音频)延时之间的延时差异。
因而，在本发明的解码方法中，延时差异不是通过在音频路径或视频路径中***性地增加偏移来补偿，而是通过利用PTS和PCR之间的时间差(如果它是足够的)以提前回放有关(音频或视频)样本来补偿。用户因而没有经受***性偏移的影响，并且用户体验因而得到相当大的改善。
本发明可以鉴于以下对于本发明的特定非限定性实施例的描述而被更好地理解。
附图说明
参考附图，其中：
图1示出了现有技术解码器盒、扬声器、以及电视机，音频路径的音频延时大于视频路径的视频延时；
图2是类似于图1的附图，视频延时大于音频延时；
图3示出了在其中执行本发明的解码方法的解码器盒、扬声器、以及电视机，音频延时大于视频延时；
图4示出了本发明的解码方法的诸步骤，如在图3的配置中执行的；
图5是类似于图3的附图，视频延时大于音频延时；
图6示出了本发明的解码方法的诸步骤，如在图5的配置中执行的。
具体实施方式
参考图3，本发明的解码方法是在连接到音频回放装备(具体而言是扬声器12)和视频回放装备(具体而言是电视机13)的解码器盒11中执行的。
解码器盒11接收运送基准时钟信号并包括音频样本和视频样本的传入的音频/视频流，诸样本具有相应的演示时间戳。
解码器盒11包括音频解码器模块14和音频捕获模块15。音频路径包括音频解码器模块14、音频捕获模块15和扬声器12。
解码器盒11还包括视频解码器模块16。视频路径包括视频解码器模块16和电视机13。
解码器盒11还包括用于将音频样本和视频样本这两者相对于基准时钟信号进行同步的同步机制17。
在图3的示例中，假定扬声器12在音频路径中引入90ms的延时。音频捕获模块15在音频路径中引入80ms的延时。总音频延时因而是170ms。
电视机13在视频路径中引入70ms的延时。总视频延时因而是70ms。
音频延时因而大于视频延时。
在传入的音频/视频流中，音频样本和视频样本是提前接收的以便被解码。这一提前由诸样本的演示时间戳和基准时钟信号之间的时间差来限定。在实践中，对于音频，这一提前可在100ms到1秒(s)的范围中。对于视频，这一提前因为必需的缓冲而更小。
在图3的示例中，音频样本的时间差Δt_a是200ms，而视频样本的时间差Δt_v是50ms。
代替在视频路径中***性地引入正视频解码延迟来补偿音频路径和视频路径之间的延时差异，利用这一提前来迫使音频样本更早离开同步机制以使它们更快地供用于回放。
本发明的解码方法因而包括估算由音频路径引入的音频延时(在该示例中等于170ms)和由视频路径引入的视频延时(在该示例中等于70ms)的初始步骤。
此后并参考图4，该解码方法包括测试第一条件以确保音频样本的演示时间戳与基准时钟信号之间的时间差足够大以使音频样本能够被正确解码的步骤(步骤E1)。
第一条件如下：
Δt_a>Δmin_a；
其中Δt_a是时间差，且Δmin_a是能够正确解码音频的最小历时。
Δmin_a的值是取决于编解码器的类型、数据率等的常数。它是通过在流集合上进行测量来确定的：其历时降低，直至音频不再能由解码器盒11正确播放为止。
如果不满足第一条件，则在视频解码器模块16执行视频解码时通过增加正视频解码延迟来完全补偿延时差异(步骤E2)。
由此：
Delay_v＝Δlat_a；
Offset_a＝0；
其中Delay_v是在视频解码器模块16执行视频解码时引入的正视频解码延迟，并且其中Δlat_a是延时差异(在该示例中等于100ms)。
Offset_a＝0意味着没有施加音频偏移，并且音频样本的演示时间戳没有降低。
这一情形对应于常规情形，没有执行本发明。
如在该示例中那样，如果满足第一条件，则该解码方法包括测试第二条件以估算延时差异是否能通过降低音频样本的演示时间戳来被完全补偿的步骤(步骤E3)。
第二条件如下：
Δt_a-Δmin_a>Δlat_a。
如果不满足第二条件，则延时差异通过以下两者来被补偿：
·在执行音频解码时引入负音频偏移Offset_a，以降低音频样本的演示时间戳，使得：
Offset_a＝Δmin_a-Δt_a；以及
·在执行视频解码时增加正视频解码延迟Delay_v，使得：
Delay_v＝Δlat_a+Offset_a(步骤E4)。
相反，如在该示例中那样，如果满足第二条件，则在执行音频解码时通过引入负音频偏移Offset_a以降低音频样本的演示时间戳，延时差异被完全补偿，使得：
Offset_a＝-Δlat_a(步骤E5)。
这一情形对应于图3的示例，其中延时差异由等于-100ms的负音频偏移Offset_a完全补偿。
以下也适用：
Delay_v＝0，这意味着没有在视频路径中引入视频解码延迟。
因而，代替***性地引入正视频解码延迟，本发明的解码方法利用时间差PTS-PCR(在它足够大时)来降低音频样本的原始时间戳并通过更快地将它们回放来对延时进行补偿。用户不再经受偏移的后果的影响，并且用户体验得到改进。在本发明中，同步机制17被用来对样本的发送进行偏移。
所实现的解决方案因而包括利用传入的音频/视频流以加速传送，代替***性地引入偏移(如在现有技术中完成的)。
这一解决方案尤其适用于具有相异的音频路径和视频路径的***。这因以下事实而是可能的：解码方法是在解码器盒11中执行的，其接收音频流和视频流两者以用于将它们转发给扬声器12和电视机13，并且其因此控制传入的音频/视频流并能操作它。
参考图5，本发明的解码方法再次在解码器盒11中执行，解码器盒11连接到扬声器12和电视机13。
在图5的示例中，假定扬声器12在音频路径中引入90ms的延时。音频捕获模块15在音频路径中引入80ms的延时。总音频延时因而是170ms。
电视机13在视频路径中引入190ms的延时。总视频延时因而是190ms。
视频延时因而大于音频延时。
在传入的音频/视频流中，音频样本和视频样本是提前接收的以便被解码。这一提前由诸样本的演示时间戳和基准时钟信号之间的时间差来限定。
在图5的示例中，音频样本的时间差Δt_a是+200ms，而视频样本的时间差Δt_v是+50ms。
代替在音频路径中***性地引入正音频解码延迟来补偿音频路径和视频路径之间的延时差异，利用这一提前来迫使视频样本更早离开同步机制以使它们更快地供用于回放。
本发明的解码方法因而包括估算由音频路径引入的音频延时(在该示例中等于170ms)和由视频路径引入的视频延时(在该示例中等于190ms)的初始步骤。
此后并参考图6，该解码方法包括测试第一条件以确保视频样本的演示时间戳与基准时钟信号之间的时间差足够大以使视频样本能够被正确解码的步骤(步骤E11)。
第一条件如下：
Δt_v>Δmin_v，其中Δt_v是时间差，且Δmin_v是能够正确解码视频的最小历时。
Δmin_v的值是取决于编解码器的类型、数据率等的常数。它是通过在流集合上进行测量来确定的：其历时降低，直至视频不再能由解码器盒11正确播放为止。
如果不满足第一条件，则在音频解码器模块14执行音频解码时通过增加正音频解码延迟来完全补偿延时差异(步骤E12)。
由此：
Delay_a＝Δlat_v；
Offset_v＝0；
其中Delay_a是在音频解码器模块14执行音频解码时引入的正音频解码延迟，并且其中Δlat_v是延时差异(在该示例中等于20ms)。
Offset_v＝0意味着没有施加视频偏移，并且视频样本的演示时间戳没有降低。
这一情形对应于常规情形，没有执行本发明。
如在该示例中，如果满足第一条件，则该解码方法包括测试第二条件以估算延时差异是否能通过降低视频样本的演示时间戳来被完全补偿的步骤(步骤E13)。
第二条件如下：
Δt_v-Δmin_v>Δlat_v。
如果不满足第二条件，则延时差异通过以下两者来被补偿：
·在执行视频解码时引入负视频偏移Offset_v，以降低视频样本的演示时间戳，使得：
Offset_v＝Δmin_v-Δt_v；以及
·在执行音频解码时增加正音频解码延迟Delay_a，使得：
Delay_a＝Δlat_v+Offset_v(步骤E14)。
相反，如在该示例中，如果满足第二条件，则在执行视频解码时通过引入负视频偏移Offset_v以降低视频样本的演示时间戳，延时差异被完全补偿，使得：
Offset_v＝-Δlat_v(步骤E15)。
这一情形对应于图5的示例，其中延时差异由等于-20ms的负视频偏移Offset_v完全补偿。
以下也适用：
Delay_a＝0，这意味着没有在音频路径中引入音频解码延迟。
自然地，本发明不限于所描述的实施例，而是涵盖了落入如由权利要求书限定的本发明范围内的任何变型。
具体而言，连接到解码器盒的回放装备不必一定包括扬声器和电视机。

Claims (10)

1.一种用于解码由连接到音频回放装备(12)和视频回放装备(13)的解码器盒(11)接收到的传入的音频/视频流的解码方法，所述传入的音频/视频流运送基准时钟信号并包括包含相应演示时间戳的音频样本和视频样本，所述解码方法包括以下步骤：
·估算由音频路径引入的音频延时和由视频路径引入的视频延时；
·如果音频(或相应地视频)延时大于视频(或相应地音频)延时，则测试第一条件以确保音频(或相应地视频)样本与所述基准时钟信号之间的时间差(Δt_a，Δt_v)足以使得能够正确地解码所述音频(或相应地视频)样本，并且如果满足第一条件，则在解码音频(或相应地视频)时采取操作以降低所述音频(或相应地视频)样本的演示时间戳以提前回放所述音频(或相应地视频)样本，由此至少部分地补偿所述音频(或相应地视频)延时和所述视频(或相应地音频)延时之间的延时差异(Δlat_a，Δlat_v)。
2.如权利要求1所述的解码方法，其特征在于，所述第一条件如下：
Δt>Δmin，
其中Δt是所述时间差，且Δmin是用于使所述音频(或相应地视频)能够被正确解码的最小历时。
3.如权利要求2所述的解码方法，其特征在于，如果不满足所述第一条件，则在执行所述视频(或相应地音频)解码时通过增加正视频(或相应地音频)解码延迟来完全补偿所述延时差异。
4.如权利要求2所述的解码方法，其特征在于，如果满足所述第一条件，则所述解码方法包括测试第二条件以估算所述延时差异是否能通过降低所述音频(或相应地视频)样本的演示时间戳来被完全补偿的步骤。
5.如权利要求4所述的解码方法，其特征在于，所述第二条件如下：
Δt-Δmin>Δlat，
其中Δlat是所述延时差异。
6.如权利要求4所述的解码方法，其特征在于，如果不满足所述第二条件，则通过以下操作来补偿所述延时差异：
·在执行音频(或相应地视频)解码时引入负音频(或相应地视频)偏移，以降低所述音频(或相应地视频)样本的演示时间戳，使得：
Offset＝Δmin-Δt；
·在执行视频(或相应地音频)解码时增加正视频(或相应地音频)解码延迟，使得：
Delay＝Δlat+Offset。
7.如权利要求4所述的方法，其特征在于，如果满足所述第二条件，则在执行所述音频(或相应地视频)解码时通过引入负音频(或相应地视频)偏移以降低所述音频(或相应地视频)样本的演示时间戳来完全补偿所述延时差异，使得：
Offset＝-Δlat。
8.一种包括音频解码器模块和视频解码器模块的解码器盒，如任一前述权利要求所述的解码方法在所述解码器盒中执行。
9.一种包括指令的计算机程序，所述指令使得如权利要求8所述的解码器盒执行如权利要求1到7中的任一项所述的解码方法。
10.一种其上存储有如权利要求9所述的计算机程序的计算机可读存储介质。
CN202010914890.2A 2019-09-04 2020-09-03 解码传入的音频/视频***的方法 Pending CN112449233A (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1909729 2019-09-04
FR1909729A FR3100412B1 (fr) 2019-09-04 2019-09-04 Procédé de décodage d’un flux d’entrée audio/vidéo

Publications (1)

Publication Number Publication Date
CN112449233A true CN112449233A (zh) 2021-03-05

Family

ID=68987915

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010914890.2A Pending CN112449233A (zh) 2019-09-04 2020-09-03 解码传入的音频/视频***的方法

Country Status (5)

Country Link
US (1) US11756590B2 (zh)
EP (1) EP3790282B1 (zh)
CN (1) CN112449233A (zh)
BR (1) BR102020018100A2 (zh)
FR (1) FR3100412B1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023015404A1 (zh) * 2021-08-09 2023-02-16 深圳Tcl新技术有限公司 音频播放方法、装置、电子设备及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114710687B (zh) * 2022-03-22 2024-03-19 阿里巴巴（中国）有限公司 音视频同步方法、装置、设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10356143B2 (en) * 2012-10-10 2019-07-16 Samsung Electronics Co., Ltd. Method and apparatus for media data delivery control
US9826015B2 (en) * 2013-09-04 2017-11-21 Qualcomm Incorporated Dynamic and automatic control of latency buffering for audio/video streaming
WO2016060474A1 (ko) * 2014-10-14 2016-04-21 엘지전자(주) Hdmi를 사용하여 데이터를 송수신하기 위한 방법 및 장치
US20170006331A1 (en) * 2015-06-30 2017-01-05 Stmicroelectronics International N.V. Synchronized rendering of split multimedia content on network clients
EP3324638A1 (en) * 2016-11-16 2018-05-23 Thomson Licensing Method for decoding an audio/video stream and corresponding device
US10200768B2 (en) * 2017-03-30 2019-02-05 Microsoft Technology Licensing, Llc Low-latency mobile device audiovisual streaming

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023015404A1 (zh) * 2021-08-09 2023-02-16 深圳Tcl新技术有限公司 音频播放方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
EP3790282B1 (fr) 2023-04-05
EP3790282A1 (fr) 2021-03-10
BR102020018100A2 (pt) 2021-03-16
FR3100412B1 (fr) 2021-08-06
US11756590B2 (en) 2023-09-12
US20210065749A1 (en) 2021-03-04
FR3100412A1 (fr) 2021-03-05

Similar Documents

Publication Publication Date Title
US8199780B2 (en) Multimedia apparatus and synchronization method thereof
KR101535827B1 (ko) 무선 통신 네트워크에서 오디오 및 비디오 동기화 장치 및 방법
US8363161B2 (en) Systems, methods, and apparatus for synchronization of audio and video signals
US8620134B2 (en) Video and audio reproducing apparatus and video and audio reproducing method for reproducing video images and sound based on video and audio streams
US20140376873A1 (en) Video-audio processing device and video-audio processing method
KR101967818B1 (ko) 수신 장치 및 그 수신 방법
KR101168612B1 (ko) 디지털 방송수신기의 동기장치 및 방법
JP5660895B2 (ja) ユーザー観察可能な信号を同期化させるための装置および方法
KR20050085344A (ko) 신호 동기화 방법 및 시스템
JP2008506282A (ja) インターネットプロトコルに用いるストリーミングオーディオとストリーミングビデオとの同期保持
US9723180B2 (en) Device and method for correcting lip sync problems on display devices
KR20120126007A (ko) 영상 전송 장치, 영상 표시 장치, 영상 표시 시스템, 영상 전송 방법 및 컴퓨터 프로그램
CN112449233A (zh) 解码传入的音频/视频***的方法
CN101106724A (zh) 将视频数据及音频数据同步的方法及***
JP4468001B2 (ja) ビデオデータと補助データの読み出し同期装置及び処理及び関連製品
CN108810656B (zh) 一种实时直播ts流的去抖处理方法及处理***
JPWO2006064689A1 (ja) 無線通信システム
KR20140018984A (ko) 다중 모듈로부터 동기화된 사용자 경험을 제공하기 위한 방법 및 시스템
US11743561B2 (en) Image display device, lip-sync correction method thereof, and image display system
KR20060105890A (ko) 디지털 방송 수신 장치 및 그 장치에서의 동기화 처리 방법
EP1889488B1 (en) Audio-video processing system
JP2009049919A (ja) 映像音声再生方法および映像音声再生システム
US20100166382A1 (en) Video and audio reproduction system, distribution device, and synchronization adjustment method
KR100698182B1 (ko) 디지털 방송 시스템의 ａｖ 출력 방법 및 장치
CN112333610B (zh) 一种蓝牙tws设备的音频播放方法及装置

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination