TWI475842B - Real-time control method of servo-to-client data stream transfer rate - Google Patents
Real-time control method of servo-to-client data stream transfer rate Download PDFInfo
- Publication number
- TWI475842B TWI475842B TW095100616A TW95100616A TWI475842B TW I475842 B TWI475842 B TW I475842B TW 095100616 A TW095100616 A TW 095100616A TW 95100616 A TW95100616 A TW 95100616A TW I475842 B TWI475842 B TW I475842B
- Authority
- TW
- Taiwan
- Prior art keywords
- client
- data
- video
- server
- packet
- Prior art date
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Small-Scale Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Description
本發明有關多媒體資料之傳輸，尤其有關區域網路/廣域網路環境中連續資料即時傳輸之方法與系統。The invention relates to the transmission of multimedia materials, in particular to a method and system for real-time transmission of continuous data in a regional network/wide area network environment.
本申請案為美國申請案第11/172,489號，發明名稱「用於網路/廣域網路環境中連續媒體流之自適應媒體傳輸管理」之延續申請案，該案係於2005年6月29日由Huang提出申請，其中請求2004年6月29日提出之美國暫時申請案第60/583,360號之權益，並全案併入本案。This application is a continuation application of U.S. Application No. 11/172,489, entitled "Adaptive Media Transmission Management for Continuous Media Streaming in a Network/WAN Environment", which was filed on June 29, 2005. An application was filed by Huang, which requested the right of US Provisional Application No. 60/583,360, filed on June 29, 2004, and the case was incorporated into the case.
隨著種種新應用方法與新使用模式的出現，區域網路/廣域網路環境下流動的視訊已獲得顯著的重要性。消費者可以從桌上型或膝上型電腦享受觀賞視訊流及/或聆聽音訊流的樂趣，從傳統媒體的耗用中創造一種全新的典型。不但在個人數位視訊娛樂的環境中，視訊流也已隨著視訊會議及遠距離教學的日漸普及而在企業環境中獲得接受。With the emergence of new application methods and new usage models, video streaming in the regional/wide area network environment has gained significant importance. Consumers can enjoy watching video streams and/or listening to audio streams from desktop or laptop computers, creating a new paradigm from the consumption of traditional media. Not only in the personal digital video entertainment environment, video streaming has been accepted in the corporate environment with the increasing popularity of video conferencing and distance learning.
不幸的是，視訊流的普及化對於現有網路基礎建設的可用帶寬，已造成極大的負擔。在已往以網路為基礎的各種應用中，具有意義的資料可以用一種爆發模式(burst mode)傳送或接收。例如，使用網路瀏覽器的使用者可以在一URL(單一資源定址器)鏈結上按滑鼠鍵來呼叫喚用(invoking)一網址，以更新(update)使用者之網頁。網路瀏覽器與所要的網址間的網路通信，可能持續到網頁更新完成或使用者選取另一URL更新(refresh)其瀏覽器為止。然而，與已往以網路為基礎的各種應用不同的是，流動的影音資料在客戶端與伺服端之間的連線建立後，就必須以連續流的方式送達一客戶(觀看者)。若資料流的數量未維持在某一位準以上時(或未等於一最低輸貫量時)，視訊可能會因為解碼器沒有足夠的資料可處理(解碼)而出現停止狀況。但等資料再度可用時，視訊會立即回復。Unfortunately, the popularity of video streaming has placed a significant burden on the available bandwidth of existing network infrastructure. In various web-based applications, meaningful data can be transmitted or received in a burst mode. For example, a user using a web browser can invoke a URL on a URL (single resource locator) link to invoking a web address to update the user's web page. Network communication between the web browser and the desired web address may continue until the web page update is complete or the user selects another URL to refresh their browser. However, unlike previous web-based applications, streaming video and audio data must be delivered to a client (viewer) in a continuous stream after the connection between the client and the server is established. If the number of streams is not maintained above a certain level (or not equal to a minimum amount of transmission), the video may be stopped because the decoder does not have enough data to process (decode). However, the video will respond immediately when the data is available again.
已往的視訊廣播是在類比網域中傳送視頻資訊，流動的視訊資訊也與已往的視訊廣播不同，它必須先轉換成數位格式，因此成為一IP資料封包(IP packets)，才能在一網際網路通信協定(IP)網路[諸如家庭網路(區域網路)或網際網路(廣域網路)]上傳送。然而，視訊從類比網域簡單轉換成數位網域，並無法立即讓使用者可以在IP網路上傳輸視訊，因為所需的視訊帶寬大於多數現有網路允許的帶寬。例如，(根據國家電視標準委員會)以標準的每秒30幀的速率放映，且像素達720x480的標準家用電視而言，其中每一像素需要2位元組作彩色放映，若要傳輸一全螢幕視訊而不需任何資料壓縮時，其最小輸貫量(throughput)將在19.78MBps或158Mbps左右。此一條件顯然超過一典型區域網路之容量(10/100Mbps)所能支援的範圍。因此，透過習用網路架構傳送原始視訊總長並不實際；或者，至少很難用現有技術在網路上作現場視訊傳輸。因此，在IP網路上傳輸資料前壓縮視訊資料，成為一種必要條件。In the past, video broadcasting was to transmit video information in the analog domain. The streaming video information was different from the previous video broadcasting. It had to be converted into a digital format, so it became an IP packet. A communication protocol (IP) network [such as a home network (regional network) or an internet (wide area network)]. However, the simple conversion of video from an analog domain to a digital domain does not immediately allow users to transmit video over an IP network because the required video bandwidth is greater than the bandwidth allowed by most existing networks. For example, (according to the National Television Standards Committee) at a standard rate of 30 frames per second, and a standard home TV with 720x480 pixels, each pixel requires 2 bytes for color projection, to transmit a full screen. For video without any data compression, the minimum throughput will be around 19.78MBps or 158Mbps. This condition obviously exceeds the range supported by the capacity of a typical local area network (10/100 Mbps). Therefore, it is not practical to transmit the original video length through the conventional network architecture; or, at least, it is difficult to use the existing technology for live video transmission on the network. Therefore, compressing video data before transmitting data on an IP network becomes a necessary condition.
目前已有多種視訊壓縮技術推出，並不斷在改進中。其中最令人注意的幾種壓縮技術是MPEG、H.261、H.263、及H.264。然而，即使有這些壓縮引擎，仍需要恆定的輸貫量，才能將流動視訊從伺服端傳送到一顯示客戶端。不幸的是，區域/廣域網路是設計成一種「盡力型」媒介，意即它雖然試圖在最低可能之時間範圍內使資料從來源端(伺服端)抵達接收端(客戶端)，但是並不能保證一定如此。A variety of video compression technologies have been introduced and are constantly being improved. The most notable compression techniques are MPEG, H.261, H.263, and H.264. However, even with these compression engines, a constant amount of throughput is required to transfer mobile video from the server to a display client. Unfortunately, the WAN/WAN is designed as a “best effort” medium, meaning that although it attempts to get data from the source (server) to the receiver (client) in the lowest possible time range, it does not. This must be guaranteed.
若未滿足最低輸貫量，在播放時可能會出現視訊跳動。為了找出流動視訊中既有的視訊跳動，各種線上視訊(VOD)應用中廣泛使用不同方法，諸如在客戶端及伺服端都配置緩衝器。典型線上視訊應用中的內容提供者(等於伺服端)可於顯示客戶端因傳送中產生封包漏失或轉訛而要求時，重新傳輸視訊資料。此種技術可除去視訊跳動並增強觀賞感受。然而，此種方法雖然達到線上視訊的目的，但在需要即時觀賞的應用中，可能引發顯著的延遲。某些即時應用中，諸如現場視訊轉播、保全監看、及視訊會議，具有時間敏感性，在此情況下，伺服端只能傳送視訊資料一次。If the minimum amount of transmission is not met, video jitter may occur during playback. In order to find out the existing video jitter in mobile video, different methods are widely used in various online video (VOD) applications, such as configuring buffers on both the client and the server. The content provider (equivalent to the server) in a typical online video application can retransmit the video data when the display client requests that the packet is lost or transferred during transmission. This technology removes video jitter and enhances viewing experience. However, while this approach achieves the goal of online video, it can cause significant delays in applications that require instant viewing. In some instant applications, such as live video broadcast, security monitoring, and video conferencing, it is time sensitive. In this case, the server can only transmit video data once.
為了能克服網路的「盡力型」特性，並同時以更有效更具可預測性的方式管理網路上的通信量，現今已發展出多種不同的服務品質(QoS)處理方法。例如，其中一種方法是，可將一優先旗標放在資訊流的標頭中，以允許一流動應用從路由器及其他應用接收較高的服務水準。此種方法係用DiffServ表示之。另一種方法是，允許流動應用經由一保留請求將其服務要件廣播給網路。此種方法係用IntServ表示之。通常，根據服務品質知悉型(QoS－awared)路由器的流動應用，可以使用資源保留設置協定(RSVP)作為其端對端信號協定。不幸的是，這些方法都不能保證伺服端與客戶端之間的每一轉送點都有支援或理解這些協定的設備，因此，有可能無法達成那些任務重大型應用所需的連續資料流要件。In order to overcome the "best effort" nature of the network while managing traffic on the network in a more efficient and predictable manner, a number of different quality of service (QoS) processing methods have been developed. For example, one method is to place a priority flag in the header of the information stream to allow a mobile application to receive a higher level of service from the router and other applications. This method is represented by DiffServ. Another approach is to allow the mobile application to broadcast its service requirements to the network via a reservation request. This method is represented by IntServ. In general, according to the mobile application of a QoS-awared router, the Resource Reservation Setting Protocol (RSVP) can be used as its end-to-end signal protocol. Unfortunately, none of these methods guarantees that each transfer point between the server and the client has a device that supports or understands these protocols, so it may not be possible to achieve the continuous stream elements required for mission-critical applications.
因此，吾人亟需一種系統，既可跨越傳統網路以即時方式傳輸資料的系統，並可同時在資料傳輸期間滿足最低閾要件，以維持人類視覺可接受的視訊品質。Therefore, we need a system that can transmit data in real time across traditional networks, and at the same time meet minimum thresholds during data transmission to maintain human visually acceptable video quality.
本發明提供多種可跨越傳統網路或通道以即時傳輸已壓縮影音資料流的方法及系統。每一系統均使用自適應速率控制(Adaptive Rate Control,ARC)技術。此種技術係以偵測瑕疵封包及其他諸如整體系統效能、使用模式、及通道特性等影響性因素為基礎。為了控制各通道中的資料流壅塞，並將目視顯示品質維持在某一水準以上，本發明進而有效利用人類的視覺及人類對變動的目視狀態的適應力。The present invention provides a variety of methods and systems for instantaneously transmitting compressed video data streams across conventional networks or channels. Each system uses Adaptive Rate Control (ARC) technology. This technology is based on the detection of 瑕疵 packets and other influencing factors such as overall system performance, usage patterns, and channel characteristics. In order to control the flow choking in each channel and maintain the visual display quality above a certain level, the present invention further utilizes human vision and human adaptability to changing visual conditions.
前述自適應速率控制(ARC)技術有賴於客戶端對伺服端所傳送視訊封包品質的計算能力，以及對伺服端傳送所需傳輸率資訊的能力。此種方法簡化了已往伺服端或內容提供端所承擔的硬體與軟體實施工具複雜性，並將監看帶寬與傳輸控制的負擔從伺服端移轉到客戶端，藉此減少整體設計成本。The aforementioned adaptive rate control (ARC) technology relies on the client's ability to calculate the video packet quality transmitted by the server and the ability to transmit the required transmission rate information to the server. This method simplifies the hardware and software implementation complexity of the previous server or content provider, and shifts the monitoring bandwidth and transmission control burden from the server to the client, thereby reducing the overall design cost.
本發明目的之一係提供一種伺服端至客戶端資料流傳輸率之即時控制方法，其中，該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟：以一傳輸率從伺服端接收一資料流的第一部份，其中該第一部份包括多數資料封包；檢查該等資料封包之完整性；根據其完整性決定一封包漏失比；使用該封包漏失比及該比對檢查表產生一調整該傳輸率之要求；將該要求傳送至該伺服端；以及，以調整後的傳輸率從該伺服端接收該資料流的第二部份。One of the objects of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of characteristics thereof. The method comprises the steps of: receiving a first portion of a data stream from a server at a transmission rate, wherein the first portion comprises a majority data packet; checking the integrity of the data packet; determining a packet based on its integrity a leakage ratio; using the packet leakage ratio and the comparison checklist to generate a request for adjusting the transmission rate; transmitting the request to the server; and receiving the data stream from the server at the adjusted transmission rate Two parts.
本發明另一目的係提供一種伺服端至客戶端資料流傳輸率之即時控制方法，其中，該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟：接收客戶端對調整傳輸率的要求，其中該客戶端係使用一封包漏失比及該比對檢查表產生該項要求；接收至少一筆音頻輸入及至少一筆視訊輸入；根據該項要求更新一資料率及該傳輸率；以更新後的資料率編碼該音頻輸入及該視訊輸入；將編碼後音頻及視訊輸入複合成該資料流內；以及，將該資料流傳輸至該客戶端。Another object of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of features thereof. The method includes the steps of: receiving a request from a client to adjust a transmission rate, wherein the client generates the request by using a packet loss ratio and the comparison checklist; receiving at least one audio input and at least one video input; Requires updating a data rate and the transmission rate; encoding the audio input and the video input at an updated data rate; composing the encoded audio and video input into the data stream; and transmitting the data stream to the client end.
本發明再一目的係提供一種伺服端至客戶端資料流傳輸率之即時控制方法，其中，該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟：使該伺服端以一傳輸率將一資料流之第一部份傳輸至該客戶端；使該客戶端接收該第一部份，其中該第一部份包括多數資料封包；使該客戶端檢查該等資料封包之完整性，並根據其完整性決定一封包漏失比；使該客戶端使用該封包漏失比及該比對檢查表產生一調整該傳輸率之要求，並將該要求傳送至該伺服端；使該伺服端接收該項要求並根據該項要求調整傳輸率；以及，使該伺服端以調整後的傳輸率傳輸該資料流的第二部份。Still another object of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of features thereof. The method includes the steps of: causing the server to transmit a first portion of a data stream to the client at a transmission rate; causing the client to receive the first portion, wherein the first portion includes a majority data packet Causing the client to check the integrity of the data packet and determine a packet loss ratio based on its integrity; causing the client to use the packet loss ratio and the comparison checklist to generate a request to adjust the transmission rate, and Transmitting the request to the server; causing the server to receive the request and adjusting the transmission rate according to the requirement; and causing the server to transmit the second portion of the data stream at the adjusted transmission rate.
藉由參照以下圖式、說明及申請專利範圍，可以更加了解本發明上述及其他特點、目的及優點。The above and other features, objects and advantages of the present invention will become more apparent from the <RTIgt;
以下詳細說明係本發明目前最佳實施模式，其目的並非用以限制本發明，僅係說明本發明之一般原理，因為本發明範圍係由所附申請專利範圍充分定義之。The present invention is described in the following detailed description of the preferred embodiments of the invention.
首先必須指出，本說明書及所附申請專利範圍中使用的單數形式「一」、「及」、「該」，除非上下文義中另有明確表示，否則係包括多數。因此，例如，提及「一客戶端」時，係包括一或多個客戶端及熟悉此類技術之人士所知的同等意義。It must be noted that the singular forms "a", "and" and "the" are used in the <RTIgt; Thus, for example, reference to "a client" includes the equivalent meaning of one or more clients and those skilled in the art.
廣義而言，本發明提供多種在即時環境中跨越多數傳統網路或通道而傳輸壓縮影音資料流的方法與系統。每一系統可使用自適應速率控制(Adaptive Rate Control,ARC)技術。此種技術係以偵測瑕疵封包及其他諸如整體系統效能、使用模式、及通道特性等影響性因素為基礎。本發明與現行僅根據通道特徵或多層流研究法的壅塞控制機制不同，它可有效利用人類的視覺及人類對變動的目視狀態的適應力，將目視顯示品質維持在一可接受的水準之上。Broadly speaking, the present invention provides a variety of methods and systems for transmitting compressed video data streams across most conventional networks or channels in an instant environment. Adaptive Rate Control (ARC) technology can be used for each system. This technology is based on the detection of 瑕疵 packets and other influencing factors such as overall system performance, usage patterns, and channel characteristics. The present invention is different from the current occlusion control mechanism based only on the channel characteristics or the multi-layer flow research method, and can effectively utilize the human visual and human adaptability to the changing visual state to maintain the visual display quality at an acceptable level. .
前述自適應速率控制(ARC)技術可依賴客戶端對伺服端所傳送視訊封包品質的計算能力，以及傳送所需傳輸率資訊給伺服端的能力。此種方法簡化了已往伺服端或內容提供端所承擔的硬體/軟體實施工具複雜性，並將監看帶寬與傳輸控制的負擔從伺服端移轉到客戶端，藉此減少整體設計成本。The aforementioned adaptive rate control (ARC) technology can rely on the computing power of the client to transmit the video packet quality of the server, and the ability to transmit the required transmission rate information to the server. This method simplifies the hardware/software implementation tool complexity that has been undertaken by the previous server or content provider, and shifts the monitoring bandwidth and transmission control burden from the server to the client, thereby reducing the overall design cost.
請參照圖1。圖1係根據本發明之即時影音資料傳輸系統100之示意圖，該系統包括一區域網路(LAN)系統101及一廣域網路(WAN)系統121。如圖所示，區域網路101係指空中120左側的網路系統，它可包括一網路纜線或WiFi通道130(以下簡稱纜線)，多數與纜線130連接之個人電腦114、116，一與纜線130連接之伺服端106，一與纜線130連接之路由器112，及一架構為可與路由器112通信之膝上型電腦110。區域網路系統101亦可包括多數影音顯示裝置，諸如網路攝影機108、放音機104、及電視機102。其中，該等影音顯示裝置可與伺服端106耦接，並與伺服端(有線或無線)106即時交換影音資料。區域網路系統101可透過空中120而與其他區域網路或廣域網路系統通信。所謂空中，較佳是指網際網路(Internet)。區域網路系統101中可包括一防火牆118防止諸如電腦病毒及間諜軟體等惡意的資料/程式進入系統。Please refer to Figure 1. 1 is a schematic diagram of a real-time video and audio data transmission system 100 in accordance with the present invention, the system including a local area network (LAN) system 101 and a wide area network (WAN) system 121. As shown, the local area network 101 refers to the network system on the left side of the air 120, which may include a network cable or WiFi channel 130 (hereinafter referred to as a cable), and most of the personal computers 114, 116 connected to the cable 130. A server 106 coupled to the cable 130, a router 112 coupled to the cable 130, and a laptop 110 configured to communicate with the router 112. The local area network system 101 may also include most video display devices such as a webcam 108, a player 104, and a television set 102. The audio-visual display device can be coupled to the server 106 and exchange audio and video data with the server (wired or wireless) 106 in real time. The regional network system 101 can communicate with other regional or wide area network systems over the air 120. The so-called air, preferably refers to the Internet. A firewall 118 can be included in the local area network system 101 to prevent malicious data/programs such as computer viruses and spyware from entering the system.
廣域網路係指空中120右側的系統，它可包括一組客戶端或客戶端裝置及一通常稱為熱點(Hot Spot)的存取(通達)點122，用以提供對客戶端的無線通信。其中，客戶端包括一個人電腦128及其他手持裝置，諸如個人數位助理(PDA)124及手機126。客戶端124、126、128可透過空中120而與伺服端通信。其中，伺服端可為一媒體伺服器或一內容提供者，用以即時提供流動資料給客戶端。以下提及客戶端或客戶端裝置一詞時，係指架構為可與伺服端106通信並可與伺服端106即時交換流動資料的裝置。一客戶端可具有一或多個影音顯示系統用以顯示流動資料。以下參照圖2－4進一步說明伺服端與多數客戶端之一間的資料流細節。圖1中，為方便之故，僅顯示九個客戶端102、104、108、110、114、116、124、126與128。然而，具備一般技能之人士應能了解，系統100中還可以使用其他適合類型的客戶端裝置。Wide area network refers to the system on the right side of the air 120, which may include a set of client or client devices and an access (access point) 122, commonly referred to as a hot spot, for providing wireless communication to the client. Among them, the client includes a personal computer 128 and other handheld devices, such as a personal digital assistant (PDA) 124 and a mobile phone 126. Clients 124, 126, 128 can communicate with the server over the air 120. The server can be a media server or a content provider for providing mobile data to the client immediately. References below to the term client or client device refer to a device that is configured to communicate with the server 106 and to exchange mobile data with the server 106 in real time. A client may have one or more audiovisual display systems for displaying streaming data. The details of the data flow between the server and one of the clients are further described below with reference to Figures 2-4. In FIG. 1, only nine clients 102, 104, 108, 110, 114, 116, 124, 126, and 128 are shown for convenience. However, those with general skills should be able to understand that other suitable types of client devices can be used in system 100.
圖2之方塊圖200中，顯示圖1整體所述於伺服端與多數客戶端裝置之一間的即時模式資料流。如圖所示，一伺服端202可經由一網路(區域或廣域網路)210將複合的影音輸出資料傳送給一客戶端212。以下將交互使用「複合」(multiplex)及「多元插配」(interleave)二詞。複合的影音輸出資料可包括以預定資料率(壓縮率)壓縮以減少網路210中的流動壅塞、並根據一使用者資料包協定(user datagram protocol,UDP)格式化的視頻資料。稍後會詳細說明決定資料率的方法。In block diagram 200 of FIG. 2, an instant mode data stream between the server and one of the plurality of client devices is shown in FIG. As shown, a server 202 can transmit composite audio and video output data to a client 212 via a network (area or wide area network) 210. The following will use the words "multiplex" and "interleave" interactively. The composite video output data may include video data compressed at a predetermined data rate (compression rate) to reduce flow congestion in the network 210 and formatted according to a user datagram protocol (UDP). The method of determining the data rate will be described in detail later.
當客戶端212收到複合的影音資料時，可將資料暫時儲存在一收件緩衝器214內。然後，收件緩衝器214可將資料轉傳給一影音複合分解系統218。影音複合分解系統218可將複合資料分解成二組個別的資料，其中每一組資料包含一視訊信號或一音頻信號；檢查二組資料的完整性，並將二組資料解碼。稍後會解釋，伺服端202會在傳送資料給客戶端212前，先將音頻流與視訊流多元插配成為單一資料流。如前所述，每一視訊幀可能需要相當的資料量，因此，每一幀資料在透過網路210傳送前，可先分成許多資料封包。每一封包可有一標頭，記載幀內封包總數、幀內封包編號、幀(的同位)總和檢查、及封包長度等資訊。When the client 212 receives the composite video material, the data can be temporarily stored in the recipient buffer 214. The recipient buffer 214 can then forward the data to a video composite decomposition system 218. The audio-visual composite decomposition system 218 can decompose the composite data into two sets of individual data, wherein each set of data includes a video signal or an audio signal; check the integrity of the two sets of data, and decode the two sets of data. As will be explained later, the server 202 will multiply the audio stream and the video stream into a single data stream before transmitting the data to the client 212. As mentioned above, each video frame may require a considerable amount of data. Therefore, each frame of data may be divided into a plurality of data packets before being transmitted through the network 210. Each packet can have a header, which records information such as the total number of packets in the frame, the number of intra-packets, the sum of the frames (the parity), and the length of the packet.
影音複合分解系統218有一視訊環狀緩衝器244及一音頻環狀緩衝器242(見圖4)可分別用來驗證視訊資料封包及音頻資料封包的完整性。以下會參照圖4進一步詳細說明影音複合分解系統218。當視訊環狀緩衝器244遭遇轉訛的或漏失的封包(總稱為瑕疵封包)時，它會拋棄含有瑕疵封包的幀，並對一封包錯誤累加器222傳送通知。在另一實施例中，若有轉訛的封包，可使用諸如前向錯誤更正(Forward Error Corrections)之錯誤更正法來重建該幀。在另一實施例中，若有漏失的資料(遺失的封包)，可使用諸如可逆變數長度(Reversible Variable Length)碼之復原法來處理該幀。影音複合分解系統218可將解碼後的音頻及視訊資料分別傳送給一放音器/揚聲器220a與一視訊顯示器220b。The video composite decomposition system 218 has a video ring buffer 244 and an audio ring buffer 242 (see FIG. 4) for verifying the integrity of the video data packet and the audio data packet, respectively. The video composite decomposition system 218 will be described in further detail below with reference to FIG. When the video ring buffer 244 encounters a switched or lost packet (collectively referred to as a packet), it discards the frame containing the packet and transmits a notification to the packet error accumulator 222. In another embodiment, if there is a packet that is switched, the error correction method such as Forward Error Corrections can be used to reconstruct the frame. In another embodiment, if there is missing data (missing packets), the frame can be processed using a restoration method such as a Reversible Variable Length code. The audio/video composite decomposition system 218 can transmit the decoded audio and video data to a speaker/speaker 220a and a video display 220b, respectively.
封包錯誤累加器222可定期決定與傳送一封包漏失比給一傳輸比對檢查表(Transmission LUT)224。其中，該封包漏失比之定義是：瑕疵封包計數與一時程內收到的封包總數之比。在該時程開始時，封包錯誤累加器222可重設瑕疵封包計數。封包錯誤累加器222於收到影音複合分解系統218的瑕疵封包通知時，可將瑕疵封包計數增加1。同時，它可計數於該時程內收到的封包總數。在該時程結束時，封包錯誤累加器222可計算封包漏失比，並將該比率傳送給傳輸比對檢查表224。The packet error accumulator 222 can periodically determine and transmit a packet miss ratio to a transmission LUT 224. The packet leakage ratio is defined as: the ratio of the packet count to the total number of packets received in a time interval. At the beginning of the time period, the packet error accumulator 222 can reset the packet count. The packet error accumulator 222 may increment the packet count by one when it receives the packet notification from the video composite decomposition system 218. At the same time, it counts the total number of packets received during that time period. At the end of the time period, the packet error accumulator 222 can calculate the packet leakage ratio and transmit the ratio to the transmission comparison check table 224.
該客戶端212亦可在伺服端202與客戶端212間的通信開始時，將其特徵資訊傳送給智慧型視訊傳輸管理器208。客戶端特徵資訊可包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。然後，根據這些特徵資訊，伺服端202可決定初始傳輸率，如此可使資料傳輸中的等待時間最小化，並使收到的資料品質最大化。The client 212 can also transmit its feature information to the smart video transmission manager 208 when communication between the server 202 and the client 212 begins. Client feature information may include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format type (such as MPEG1) , MPEG2, MPEG4, H.263, H.264, etc.) decoder capabilities, available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frames) Rate, data rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved). Then, based on these feature information, the server 202 can determine the initial transmission rate, which minimizes latency in data transmission and maximizes the quality of received data.
傳輸比對檢查表224可使用該封包漏失比讀取一比對檢查表，以決定目前的傳輸率是否需要調整。該比對檢查表可為客戶專用的，並使用該客戶端212的特徵來產生。然後，傳輸比對檢查表224可經由網路210將調整目前傳輸率的要求(如圖2中標示的「控制命令」)傳送給智慧型視訊傳輸管理器208。以下會參照圖4進一步說明調整的細節。The transmission alignment check table 224 can use the packet leakage ratio to read a comparison checklist to determine whether the current transmission rate needs to be adjusted. The alignment checklist can be customer specific and generated using the characteristics of the client 212. Then, the transmission comparison check table 224 can transmit the current transmission rate requirement (such as the "control command" indicated in FIG. 2) to the smart video transmission manager 208 via the network 210. Details of the adjustment will be further explained below with reference to FIG.
利用客戶端212傳送的要求及特徵資訊，智慧型視訊傳輸管理器208可以調整一組參數，包括目標幀速率、目標顯示位元率、目標解析度、目標音頻抽樣率、量化值，調整亮度色度(YUV)轉換法(4：2：2至4：2：0)、比例尺大小。管理器208亦可調整資料壓縮率(簡稱資料率)，並將調整後資料率資訊傳送給一影音複合系統204。影音複合系統204可接收一或多筆影音輸入流205，編碼接收的輸入流，使用收自智慧型視訊傳輸管理器208的資料率資訊來壓縮輸入流，將每一幀資料分成多數封包，並將此等封包傳送至一媒體存取控制(MAC)緩衝器206。如前所述，每一視訊幀可包括相當的資料量，並可分成多數資料封包，以易於透過網路210傳輸。以下會參照圖3提供有關影音複合系統204更詳細的資訊。媒體存取控制(MAC)緩衝器206可暫時包含欲透過網路210傳輸的封包，並視客戶端的數目而定，以單聲道調頻廣播(unicast)、立體聲雙聲道調頻廣播(multicast)、或廣播(broadcast)模式傳送此等封包。Using the requirements and feature information transmitted by the client 212, the smart video transmission manager 208 can adjust a set of parameters including target frame rate, target display bit rate, target resolution, target audio sampling rate, quantized value, and adjust the brightness color. Degree (YUV) conversion method (4:2:2 to 4:2:0), scale size. The manager 208 can also adjust the data compression rate (referred to as the data rate) and transmit the adjusted data rate information to an audio/video composite system 204. The audio/video composite system 204 can receive one or more audio and video input streams 205, encode the received input stream, compress the input stream using the data rate information received from the smart video transmission manager 208, and divide each frame of data into a plurality of packets, and These packets are passed to a Media Access Control (MAC) buffer 206. As previously mentioned, each video frame can include a substantial amount of data and can be divided into a plurality of data packets for easy transmission over the network 210. More detailed information about the audio/video composite system 204 will be provided below with reference to FIG. The Media Access Control (MAC) buffer 206 may temporarily contain packets to be transmitted over the network 210, depending on the number of clients, in mono unicast, stereo two-channel FM, and Or broadcast (broadcast) mode to transmit these packets.
圖3為圖2所示伺服端202內實施的影音複合系統204之方塊圖。如圖所示，影音複合系統204可包括一音頻編碼器230，用以編碼一音頻輸入流；一音頻環狀緩衝器232，用以轉播編碼後音頻輸入流給一影音複合器238；一視訊編碼器234，可依收自智慧型視訊傳輸管理器208的資料率編碼(或壓縮)一視訊輸入流；一視訊環狀緩衝器236，可將編碼後視訊流轉播給影音複合器238；及該影音複合器238，可將編碼後音頻流與視訊流複合成單一的複合資料流。為了維持音頻/視訊同步化及抽樣時脈完整性，影音複合器238可交織編碼後音頻流與編碼後視訊流。FIG. 3 is a block diagram of the AV composite system 204 implemented in the servo end 202 of FIG. As shown, the audio/video composite system 204 can include an audio encoder 230 for encoding an audio input stream, and an audio ring buffer 232 for relaying the encoded audio input stream to an audio/video combiner 238; The encoder 234 can encode (or compress) a video input stream according to the data rate received from the smart video transmission manager 208; a video ring buffer 236 can relay the encoded video stream to the audio/video combiner 238; The video and audio synthesizer 238 can combine the encoded audio stream and the video stream into a single composite data stream. To maintain audio/video synchronization and sample clock integrity, the video and audio synthesizer 238 can interleave the encoded audio stream with the encoded video stream.
如前所述，由一伺服端傳送的複合流封包，可能會因為網路壅塞、解碼器負載飽和、或系統負載飽和而於透過網路傳輸期間漏失或轉訛。因此，必須在分解複合流封包後，檢查透過網路接收的複合流封包之完整性。例如，圖4顯示圖2所示影音複合分解系統218之方塊圖，其中，該影音複合分解系統218可將接收的複合流分解成一音頻及一視訊流；檢查該視訊流之完整性；並將此等音頻流及視訊流解碼，以供放音器/揚聲器220a及視訊顯示器220b播放。如圖所示，影音複合分解器240可將接收的資料流解耦成一音頻流及一視訊流。然後，它可將視訊流放入視訊環狀緩衝器244，將音頻流放入一音頻環狀緩衝器242。音頻環狀緩衝器242與視訊環狀緩衝器244可分別檢查影音同步與幀同步，並於解碼前執行必要的更正。視訊環狀緩衝器244亦可檢查是否有任何轉訛及/或漏失的封包，以驗證視訊流(或封包)的完整性。當視訊環狀緩衝器244遇到瑕疵封包時，它可傳送一通知給封包錯誤累加器222。As mentioned earlier, a composite stream packet transmitted by a server may be lost or transferred during transmission through the network due to network congestion, decoder load saturation, or system load saturation. Therefore, the integrity of the composite stream packet received through the network must be checked after the composite stream packet is decomposed. For example, FIG. 4 shows a block diagram of the video composite decomposition system 218 of FIG. 2, wherein the video composite decomposition system 218 can decompose the received composite stream into an audio and a video stream; check the integrity of the video stream; These audio streams and video streams are decoded for playback by the speaker/speaker 220a and the video display 220b. As shown, the video composite resolver 240 can decouple the received data stream into an audio stream and a video stream. It then places the video stream into video ring buffer 244 and the audio stream into an audio ring buffer 242. The audio ring buffer 242 and the video ring buffer 244 can check the video sync and frame sync, respectively, and perform the necessary corrections before decoding. The video ring buffer 244 can also check for any transitions and/or missed packets to verify the integrity of the video stream (or packet). When the video ring buffer 244 encounters a packet, it can transmit a notification to the packet error accumulator 222.
如前解說，封包錯誤累加器222可決定並傳送一封包漏失比給傳輸比對檢查表224。傳輸比對檢查表224可根據該封包漏失比產生調整傳輸率的要求，並將此項要求(或控制命令)透過一使用者資料包協定(UDP)通道傳送給伺服端202。UDP命令埠/通道是一種任意埠(arbitrary port)，由伺服端202特別指定與保留，以供與客戶端212通信之用。隨後，伺服端202可藉由控制壓縮率、幀速率、圖像群組(GOP)大小、量化級別、顯示解析度、或以上各項任何組合而調整傳輸率。As previously explained, the packet error accumulator 222 can determine and transmit a packet miss ratio to the transmission alignment check table 224. The transmission comparison check table 224 can generate a transmission rate requirement according to the packet loss ratio, and transmits the request (or control command) to the server 202 through a User Datagram Protocol (UDP) channel. The UDP command port/channel is an arbitrary port that is specifically designated and reserved by the server 202 for communication with the client 212. Subsequently, the server 202 can adjust the transmission rate by controlling the compression ratio, frame rate, group of pictures (GOP) size, quantization level, display resolution, or any combination of the above.
封包漏失比可作為網路210中傳輸狀態的指示器。傳輸比對檢查表224可使用該封包漏失比讀取一比對檢查表。其中，該比對檢查表可為客戶專用的，並使用一組特徵來產生。這些特徵包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。The packet loss ratio can be used as an indicator of the state of transmission in the network 210. The transmission comparison check table 224 can use the packet missing ratio to read a comparison check list. Wherein, the comparison checklist can be customer-specific and generated using a set of features. These features include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format types (such as MPEG1, MPEG2) Decoder capabilities of MPEG4, H.263, H.264, etc., available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frame rate, data) Rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved).
傳輸狀態可細分成三種子狀態：(1)無負載狀態(2)負載狀態(3)壅塞狀態。每一子狀態可有一預定範圍的封包漏失比。例如，假設目前的傳輸狀態是在無負載狀態，其中之封包漏失比小於5%(表示傳輸速度在每秒30幀(30fps)時的最大漏失為1.5fps)。當封包漏失比小於5%時，傳輸比對檢查表224可判定目前傳輸狀態係在無負載狀態，並且可以從客戶端特徵側錄達成較高的資料率。傳輸比對檢查表224可要求伺服端202以增量方式增加傳輸率，直到封包漏失比超過30%為止，亦即，直到傳輸狀態抵達壅塞狀態為止。然後，傳輸比對檢查表224可要求伺服端202降低傳輸率，直到達到負載狀態為止(封包漏失比達到一穩定狀態)。若封包漏失比指示出，傳輸狀態已達穩定狀態，則傳輸比對檢查表224可對智慧型視訊傳輸管理器208傳送一要求，其中包括一NOP信號(無操作信號或「不動作」信號，亦即，「勿改變資料率」信號)。因為網路上的通信量概況可隨時間而變化，所以客戶端212會對伺服端202隨機傳送增加資料率的要求。The transmission state can be subdivided into three sub-states: (1) no load state (2) load state (3) congestion state. Each substate can have a predetermined range of packet leakage ratios. For example, suppose the current transmission state is in a no-load state where the packet leakage ratio is less than 5% (indicating that the maximum leakage loss at a transmission speed of 30 frames per second (30 fps) is 1.5 fps). When the packet leakage ratio is less than 5%, the transmission comparison check table 224 can determine that the current transmission state is in an unloaded state, and can achieve a higher data rate from the client feature side. The transmission alignment check table 224 may require the servo terminal 202 to incrementally increase the transmission rate until the packet leakage ratio exceeds 30%, that is, until the transmission state reaches the congestion state. Then, the transmission comparison check table 224 may require the servo terminal 202 to lower the transmission rate until the load state is reached (the packet leakage ratio reaches a steady state). If the packet loss ratio indicates that the transmission status has reached a steady state, the transmission comparison check table 224 can transmit a request to the smart video transmission manager 208, including a NOP signal (no operation signal or "no action" signal, That is, the "Do not change the data rate" signal). Since the traffic profile on the network can change over time, the client 212 will randomly transmit the request for increased data rate to the server 202.
封包漏失比的閾位準可以謹慎調整，以使客戶端212之用戶所感知的顯示品質，可隨著網路狀況變更而適度下降或改進。由於客戶端212除了其本身目前的傳輸率外，並無其他可用的帶寬通信量資訊，所以客戶端212可能會定期要求伺服端202增加傳輸率，以便可以達成較高的視訊品質。除了監看進入的影音封包流品質外，客戶端212亦可通知伺服端202顯示器尺寸是否已被顧客縮小，藉此降低傳輸資料率以匹配新的顯示器大小，並降低帶寬要件。The threshold level of the packet loss ratio can be carefully adjusted so that the display quality perceived by the user of the client 212 can be moderately reduced or improved as the network conditions change. Since the client 212 has no other available bandwidth traffic information except its current transmission rate, the client 212 may periodically request the server 202 to increase the transmission rate so that higher video quality can be achieved. In addition to monitoring the incoming video packet stream quality, the client 212 can also notify the server 202 whether the display size has been reduced by the customer, thereby reducing the transmission data rate to match the new display size and reducing the bandwidth requirements.
值得注意的是，目前用於控制傳輸率的方法，稱為自適應速率控制(ARC)技術，它可用漏失封包偵測為基礎，而非已往託即時傳輸協定(RTP)之助而以緩衝器為基礎的方法。因為自適應速率控制(ARC)技術主要可以依靠有能力計算視訊流封包品質且能對伺服端傳送所需傳輸率資訊的客戶端，所以監看帶寬與資料率控制的負擔，可以從伺服端移轉到客戶端。此外，自適應速率控制(ARC)技術亦可允許客戶端要求伺服端以最佳化的傳輸率傳送資料流，減少網路中傳輸壅塞的可能性。It is worth noting that the current method for controlling the transmission rate, called adaptive rate control (ARC) technology, can be based on lossy packet detection instead of buffering with the help of the Instant Transfer Protocol (RTP). The based approach. Because the adaptive rate control (ARC) technology can mainly rely on the client that has the ability to calculate the quality of the video stream packet and can transmit the required transmission rate information to the server, the burden of monitoring the bandwidth and data rate control can be shifted from the servo end. Go to the client. In addition, Adaptive Rate Control (ARC) technology allows the client to require the server to transmit data streams at an optimized transmission rate, reducing the possibility of traffic congestion in the network.
圖5係本發明資料傳輸率控制之典型步驟流程圖500。具有一般技能之人士可以理解，圖中所示方法可用多種方式修改而不脫離本發明精神與範圍。例如，在本發明另一實施例中，圖示方法中可結合各種不同部份、以另一種順序重新安排此等不同部份、或移除此等不同部份。此外應予注意的是，此方法可用多種不同方式實施之，諸如在一通用電腦中執行軟體、使用微處理器執行韌體及/或電腦可讀媒體、使用專門用途之硬體，等等。Figure 5 is a flow chart 500 of a typical procedure for data rate control of the present invention. It will be appreciated by those skilled in the art that the methods illustrated in the figures can be modified in various ways without departing from the spirit and scope of the invention. For example, in another embodiment of the present invention, various different portions may be combined, the different portions may be rearranged in another order, or the different portions may be removed. It should also be noted that this method can be implemented in a number of different ways, such as executing software in a general purpose computer, executing firmware and/or computer readable media using a microprocessor, using specialized hardware, and the like.
本方法可從一狀態502開始。在狀態502中，一客戶端可經由一網路(區域/廣域網路)建立與一伺服端間的連線或通道，以供資料交換。然後，在狀態504中，客戶端可在通道建立開始時，對伺服端傳送一組客戶端特徵資訊。客戶端特徵資訊可包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。然後，在狀態506中，伺服端可根據客戶端特徵資訊決定一傳輸率，並以即時模式使用該傳輸率對客戶端傳送一複合資料流。該複合資料可包括呈資料封包形式之音頻與視訊資料。然後，程序可進入狀態508。The method can begin with a state 502. In state 502, a client can establish a connection or channel with a server via a network (area/wide area network) for data exchange. Then, in state 504, the client can transmit a set of client feature information to the server at the beginning of channel establishment. Client feature information may include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format type (such as MPEG1) , MPEG2, MPEG4, H.263, H.264, etc.) decoder capabilities, available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frames) Rate, data rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved). Then, in state 506, the server can determine a transmission rate based on the client characteristic information, and transmit the composite data stream to the client using the transmission rate in an immediate mode. The composite material may include audio and video data in the form of data packets. The program can then enter state 508.
在狀態508中，客戶端可從伺服端接收該複合資料流。其次，在狀態510中，客戶端可將接收的複合資料流分解成音頻及視訊資料封包。隨後，在狀態512中，客戶端可根據瑕疵資料封包數決定視訊資料封包之封包漏失比。然後，程序可進到一決定菱形514。In state 508, the client can receive the composite stream from the server. Second, in state 510, the client can decompose the received composite data stream into audio and video data packets. Then, in state 512, the client can determine the packet loss ratio of the video data packet according to the number of data packets. The program can then proceed to a decision diamond 514.
在狀態514中，客戶端可決定該封包漏失比是否指示網路中的壅塞狀態。狀態514的答覆為肯定時，在狀態516中，客戶端可產生降低傳輸率的要求。然後，程序可進到狀態522。否則，程序可進到另一決定菱形518。In state 514, the client may determine whether the packet miss rate ratio indicates a congestion status in the network. When the answer to state 514 is affirmative, in state 516, the client may generate a request to reduce the transmission rate. The program can then proceed to state 522. Otherwise, the program can proceed to another decision diamond 518.
在狀態518中，可以決定封包漏失比是否指示網路中的無負載狀態。狀態518的回覆若為肯定時，則程序可進到狀態519。在狀態519中，可決定傳輸率是否已達最大傳輸率。狀態519的回覆若為否定時，在狀態520中，客戶端可產生增加傳輸率的要求。然後，程序可進到狀態522。In state 518, it may be determined whether the packet miss rate ratio indicates an unloaded state in the network. If the reply to state 518 is affirmative, the program can proceed to state 519. In state 519, it may be determined whether the transmission rate has reached the maximum transmission rate. If the reply to state 519 is negative, in state 520, the client may generate a request to increase the transmission rate. The program can then proceed to state 522.
狀態518的回覆為否定時，或狀態519的回覆為肯定時，則程序進到狀態521。在狀態521中，客戶端可產生一包含NOP(勿改變資料率)信號。然後，程序可進到狀態522。在狀態522中，客戶端可對伺服端傳送該項要求。隨後，在狀態524中，伺服端可用調整後的傳輸率傳輸該複合資料流；其中，調整後傳輸率是依據要求決定的。然後，程序可等到傳輸完成後進到狀態508。值得注意的是，步驟508至524應定期實施，以維持伺服端與客戶端之間建立的通道。When the reply to state 518 is negative, or if the reply to state 519 is positive, then the program proceeds to state 521. In state 521, the client can generate a signal containing a NOP (Do Not Change Data Rate). The program can then proceed to state 522. In state 522, the client can transmit the request to the server. Subsequently, in state 524, the server can transmit the composite data stream with the adjusted transmission rate; wherein the adjusted transmission rate is determined according to requirements. The program can then wait until the transfer is complete to proceed to state 508. It is worth noting that steps 508 through 524 should be implemented periodically to maintain the channel established between the server and the client.
吾人應了解前述係有關本發明之典型實施例，其中可作多種修改而不脫離本方法如下述申請專利範圍所述精神及範圍。It is to be understood that the foregoing is a description of the exemplary embodiments of the invention, and various modifications may be made without departing from the spirit and scope of the present invention.
100．．．即時影音資料傳輸系統100. . . Instant video data transmission system
101．．．區域網路系統101. . . Regional network system
102．．．電視機102. . . TV set
104．．．放音器104. . . Sounder
106．．．伺服端106. . . Servo end
108．．．網路攝影機108. . . Webcam
110．．．膝上型電腦110. . . Laptop
112．．．路由器112. . . router
114．．．個人電腦114. . . personal computer
116．．．個人電腦116. . . personal computer
118．．．防火牆118. . . Firewall
120．．．空中(網際網路)120. . . Air (internet)
121．．．廣域網路系統121. . . WAN system
122．．．通達點(熱點)122. . . Access point (hot spot)
124．．．個人數位助理124. . . Personal digital assistant
126．．．手機126. . . Mobile phone
128．．．個人電腦128. . . personal computer
130．．．網路纜線或WiFi通道130. . . Network cable or WiFi channel
200．．．伺服端與客戶端間即時模式資料流方塊圖200. . . Instant mode data flow block diagram between server and client
202．．．伺服端202. . . Servo end
204．．．影音複合系統204. . . Audio-visual composite system
205．．．影音輸入流205. . . Video input stream
206．．．媒體存取控制緩衝器206. . . Media access control buffer
208．．．智慧型視訊傳輸管理器208. . . Smart Video Transfer Manager
210．．．網路(區域/廣域)210. . . Network (region/wide area)
212．．．客戶端212. . . Client
214．．．收件緩衝器214. . . Receiving buffer
218．．．影音複合分解系統218. . . Audio-visual composite decomposition system
220a．．．放音器/揚聲器220a. . . Speaker/speaker
220b．．．視訊顯示器220b. . . Video display
222．．．封包錯誤累加器222. . . Packet error accumulator
224．．．傳輸比對檢查表224. . . Transmission comparison checklist
230．．．音頻編碼器230. . . Audio encoder
232．．．音頻環狀緩衝器232. . . Audio ring buffer
234．．．視訊編碼器234. . . Video encoder
236．．．視訊環狀緩衝器236. . . Video ring buffer
238．．．影音複合器238. . . Audio and video composite
240．．．影音複合分解器240. . . Video composite resolver
242．．．音頻環狀緩衝器242. . . Audio ring buffer
244．．．視訊環狀緩衝器244. . . Video ring buffer
500．．．資料傳輸率控制之典型步驟流程圖500. . . Flow chart of typical steps of data transmission rate control
502－524．．．資料傳輸率控制各步驟(狀態)502-524. . . Data transfer rate control steps (status)
圖1係根據本發明之即時影音資料流傳輸系統示意圖；圖2係方塊圖，說明圖1所述伺服端與客戶端之一間的資料流；圖3係圖2所示伺服端內實施之影音複合系統方塊圖；圖4係圖2所示客戶端內實施之影音複合分解系統方塊圖；及圖5係本發明資料傳輸率控制之典型步驟流程圖。1 is a schematic diagram of a real-time video and audio data stream transmission system according to the present invention; FIG. 2 is a block diagram showing the data flow between the server and the client of FIG. 1; FIG. 3 is implemented in the server shown in FIG. Block diagram of video and audio composite system; FIG. 4 is a block diagram of a video/audio composite decomposition system implemented in the client shown in FIG. 2; and FIG. 5 is a flow chart of typical steps of data transmission rate control according to the present invention.
100．．．即時影音資料傳輸系統100. . . Instant video data transmission system
101．．．區域網路系統101. . . Regional network system
102．．．電視機102. . . TV set
104．．．放音器104. . . Sounder
106．．．伺服端106. . . Servo end
108．．．網路攝影機108. . . Webcam
110．．．膝上型電腦110. . . Laptop
112．．．路由器112. . . router
114．．．個人電腦114. . . personal computer
116．．．個人電腦116. . . personal computer
118．．．防火牆118. . . Firewall
120．．．空中(網際網路)120. . . Air (internet)
121．．．廣域網路系統121. . . WAN system
122．．．通達點(熱點)122. . . Access point (hot spot)
124．．．個人數位助理124. . . Personal digital assistant
126．．．手機126. . . Mobile phone
128．．．個人電腦128. . . personal computer
130．．．網路纜線或WiFi通道130. . . Network cable or WiFi channel
Claims (20)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW095100616A TWI475842B (en) | 2006-01-06 | 2006-01-06 | Real-time control method of servo-to-client data stream transfer rate |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW095100616A TWI475842B (en) | 2006-01-06 | 2006-01-06 | Real-time control method of servo-to-client data stream transfer rate |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW200727618A TW200727618A (en) | 2007-07-16 |
| TWI475842B true TWI475842B (en) | 2015-03-01 |
Family
ID=53185879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW095100616A TWI475842B (en) | 2006-01-06 | 2006-01-06 | Real-time control method of servo-to-client data stream transfer rate |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI475842B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8532139B2 (en) * | 2007-09-21 | 2013-09-10 | Broadcom Corporation | Method and system for indicating a transition in rate and/or power consumption utilizing a distinct physical pattern on one or more idle channel(s) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5928330A (en) * | 1996-09-06 | 1999-07-27 | Motorola, Inc. | System, device, and method for streaming a multimedia file |
| US20030016673A1 (en) * | 2001-06-29 | 2003-01-23 | Ramesh Pendakur | Correcting for data losses with feedback and response |
| US20030039398A1 (en) * | 2001-08-21 | 2003-02-27 | Mcintyre Kristen A. | Dynamic bandwidth adaptive image compression/decompression scheme |
| TWI243593B (en) * | 2004-03-05 | 2005-11-11 | Chunghwa Telecom Co Ltd | Independent MPEG-2 hardware coding system with a VGA interface |
-
2006
- 2006-01-06 TW TW095100616A patent/TWI475842B/en not_active IP Right Cessation
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5928330A (en) * | 1996-09-06 | 1999-07-27 | Motorola, Inc. | System, device, and method for streaming a multimedia file |
| US20030016673A1 (en) * | 2001-06-29 | 2003-01-23 | Ramesh Pendakur | Correcting for data losses with feedback and response |
| US20030039398A1 (en) * | 2001-08-21 | 2003-02-27 | Mcintyre Kristen A. | Dynamic bandwidth adaptive image compression/decompression scheme |
| TWI243593B (en) * | 2004-03-05 | 2005-11-11 | Chunghwa Telecom Co Ltd | Independent MPEG-2 hardware coding system with a VGA interface |
Non-Patent Citations (1)
| Title |
|---|
| Jean-Chrysostome Bolot, etc., "Scalable Feedback Control for Multicast Video Distribution in the Internet", SIGCOMM 94 -8/94 London England UK, 1994 ACM. * |
Also Published As
| Publication number | Publication date |
|---|---|
| TW200727618A (en) | 2007-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7984179B1 (en) | Adaptive media transport management for continuous media stream over LAN/WAN environment | |
| TWI383684B (en) | System and method for dynamic video encoding in multimedia streaming | |
| US8446452B2 (en) | Video rate adaptation for congestion control | |
| JP4195030B2 (en) | Video data transmission method and reception method for continuous video display | |
| US7333439B2 (en) | Sending/receiving method, and device for the same | |
| US8471890B1 (en) | Adaptive video communication channel | |
| US9578179B2 (en) | Method, apparatus and system for transmitting multimedia data | |
| US7881335B2 (en) | Client-side bandwidth allocation for continuous and discrete media | |
| WO2009128528A1 (en) | Server device, content distribution method, and program | |
| US20080025414A1 (en) | Apparatus and method for transmitting moving picture stream using bluetooth | |
| US11197051B2 (en) | Systems and methods for achieving optimal network bitrate | |
| EP3016395B1 (en) | Video encoding device and video encoding method | |
| WO2020086452A1 (en) | Low-latency video internet streaming for management and transmission of multiple data streams | |
| KR20080045276A (en) | Rtp payload format for vc-1 | |
| CN114339445B (en) | Real-time video stream self-adaptive coordination system and method | |
| JP3708950B2 (en) | Transmission / reception method and apparatus | |
| US20110088069A1 (en) | Network device, information processing apparatus, stream switching method, information processing method, program, and content distribution system | |
| KR100792025B1 (en) | Method of transmitting video data in case of channel change in iptv system | |
| CN1996813B (en) | Self-adapted media transfer management of the continuous media stream used for LAN/WAN environment | |
| Fitzek et al. | Video streaming in wireless internet | |
| TWI475842B (en) | Real-time control method of servo-to-client data stream transfer rate | |
| KR101625663B1 (en) | Method and Apparatus for Receiving Content | |
| KR20050093438A (en) | Frame rate control method according to packet loss rate | |
| Zhu et al. | Research on adaptive transmission of H. 264 video stream and QoS guarantee based on SIP | |
| Sarni et al. | A novel scheme for a fast channel change in multicast IPTV system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |