TWI798840B

TWI798840B - Video data processing method and apparatus

Info

Publication number: TWI798840B
Application number: TW110135348A
Authority: TW
Inventors: 彭鵬
Original assignee: 大陸商星宸科技股份有限公司
Priority date: 2021-09-23
Filing date: 2021-09-23
Publication date: 2023-04-11
Also published as: TW202315379A

Abstract

A video data processing apparatus is provided according to one embodiment of present invention. The video data processing apparatus includes a processor and a blending circuit. The processor provides a first set of parameters according to a first channel identification signal and a second set of parameters according to a second channel identification signal. The blending circuit blends a first image into a first video data at a first time point according to the first set of parameters, and a second image into a second video data at a second time point according to the second set of parameters. The difference between the first time point and the second time point is less than a frame period of the first video data.

Description

視訊資料處理方法及裝置Video data processing method and device

本發明涉及視訊資料處理領域，具體涉及一種將圖像疊加於視訊資料的處理方法及裝置。The invention relates to the field of video data processing, in particular to a processing method and device for superimposing images on video data.

幀緩衝（Frame buffer）介面是Linux系統為顯示裝置提供的一個介面，使用者可透過此介面直接對顯示緩衝區進行讀寫操作，並且在顯示裝置上顯示。在嵌入式Linux產品中，QT、Minigui、Android Surfaceflinger、DirectFb等圖形化介面的底層都會對接幀緩衝（Frame buffer）介面，並透過幀緩衝（Frame buffer）介面可控制相關的硬體驅動層，實現顯示或圖像疊加等功能。The Frame buffer interface is an interface provided by the Linux system for the display device. Through this interface, the user can directly perform read and write operations on the display buffer and display it on the display device. In embedded Linux products, the bottom layer of graphical interfaces such as QT, Minigui, Android Surfaceflinger, and DirectFb will be connected to the frame buffer (Frame buffer) interface, and the relevant hardware driver layer can be controlled through the frame buffer (Frame buffer) interface to realize functions such as display or image overlay.

在Linux系統中，一個幀緩衝介面係透過硬體驅動層對應一個硬體，舉例來說，當透過幀緩衝介面進行圖像疊加處理時，一個幀緩衝介面係對應一個疊加電路，在此條件下，一個疊加電路僅能對一路視訊資料進行處理，當同時要處理多個視訊資料時，需要多套疊加電路才能實現，耗費相當大的硬體成本。In the Linux system, a frame buffer interface corresponds to a hardware through the hardware driver layer. For example, when image overlay processing is performed through the frame buffer interface, a frame buffer interface corresponds to an overlay circuit. Under this condition , one superimposition circuit can only process one channel of video data. When multiple video data are to be processed at the same time, multiple sets of superposition circuits are required, which consumes a considerable hardware cost.

本發明提供一種視訊資料處理方法及裝置，可透過軟體分時複用硬體以在多個視訊資料中疊加不同的圖像，以提升圖像疊加的效率及硬體的使用率。The present invention provides a video data processing method and device, which can superimpose different images in a plurality of video data by time-division multiplexing hardware through software, so as to improve the efficiency of image superposition and the utilization rate of hardware.

本發明之一實施例提供一種視訊資料處理方法，包括：提供一查找表，該查找表包含一第一通道識別信號與一第一組參數的對應關係、及一第二通道識別信號與一第二組參數的對應關係；根據該第一通道識別信號查詢該查找表，以將該第一組參數寫入一疊加電路的暫存器；利用該疊加電路，根據該第一組參數於一第一時間點將一第一圖像疊加於一第一視訊資料中；根據該第二通道識別信號查詢該查找表，以將該第二組參數寫入該疊加電路的暫存器；以及，利用該疊加電路，根據該第二組參數於一第二時間點將一第二圖像疊加於一第二視訊資料中；其中，該第一時間點與該第二時間點間的時間差小於該第一視訊資料的一幀間隔。An embodiment of the present invention provides a video data processing method, including: providing a lookup table, the lookup table includes a correspondence between a first channel identification signal and a first set of parameters, and a second channel identification signal and a first channel identification signal The corresponding relationship of the two groups of parameters; query the look-up table according to the first channel identification signal, so as to write the first group of parameters into a temporary register of a superposition circuit; use the superposition circuit, according to the first group of parameters in a first group of parameters superimposing a first image into a first video data at a time point; querying the look-up table according to the second channel identification signal, so as to write the second set of parameters into the register of the superimposing circuit; and, using the A superposition circuit, superimposing a second image on a second video data at a second time point according to the second set of parameters; wherein, the time difference between the first time point and the second time point is smaller than the first video The frame interval of the data.

本發明之另一實施例提供一種視訊資料處理裝置，包括一處理器及一疊加電路。處理器根據一第一通道識別信號提供一第一組參數，並根據一第二通道識別信號提供一第二組參數。疊加電路根據該第一組參數於一第一時間點將一第一圖像疊加於一第一視訊資料中，且根據該第二組參數於一第二時間點將一第二圖像疊加於一第二視訊資料中。該第一時間點與該第二時間點間的時間差小於該第一視訊資料的一幀間隔。Another embodiment of the present invention provides a video data processing device, including a processor and a superposition circuit. The processor provides a first set of parameters according to a first channel identification signal, and provides a second set of parameters according to a second channel identification signal. The superposition circuit superimposes a first image on a first video data at a first time point according to the first set of parameters, and superimposes a second image on a first time point at a second time point according to the second set of parameters Two video data. The time difference between the first time point and the second time point is less than a frame interval of the first video data.

下面將結合本發明實施例中的附圖，對本發明實施例中的技術方案進行清楚、完整地描述，顯然，所描述的實施例僅僅是本發明一部分實施例，而不是全部的實施例。基於本發明中的實施例，本領域技術人員在沒有作出創造性勞動前提下所獲得的所有其他實施例，都屬於本發明保護的範圍。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

本發明提出一種軟體系統架構，藉由此軟體系統架構以軟體控制一硬體電路，達成分時複用硬體電路，實現以一套硬體電路處理多個視訊資料的目的。請參考圖一，圖一是依據本發明一實施例所提出的一種軟體系統架構的示意圖。如圖一所示，軟體系統架構100包含硬體驅動層10、虛擬通道層20、業務邏輯層30、及用戶層40。軟體系統架構100可應用於一視訊資料處理裝置中。請參考圖二，圖二是依據本發明一實施例所提出的一種視訊資料處理裝置的方塊示意圖。視訊資料處理裝置200包括處理器21、影像處理電路22、圖形引擎23、疊加電路24、記憶體界面25、及記憶體26。以下係以軟體系統架構100應用於視訊資料處理裝置200的場景為例，來說明各層的作用。The present invention proposes a software system architecture, through which a hardware circuit is controlled by software to achieve time-sharing multiplexing of the hardware circuit, and realize the purpose of processing multiple video data with a set of hardware circuits. Please refer to FIG. 1 . FIG. 1 is a schematic diagram of a software system architecture according to an embodiment of the present invention. As shown in FIG. 1 , the software system architecture 100 includes a hardware driver layer 10 , a virtual channel layer 20 , a business logic layer 30 , and a user layer 40 . The software system architecture 100 can be applied in a video data processing device. Please refer to FIG. 2 . FIG. 2 is a schematic block diagram of a video data processing device according to an embodiment of the present invention. The video data processing device 200 includes a processor 21 , an image processing circuit 22 , a graphics engine 23 , an overlay circuit 24 , a memory interface 25 , and a memory 26 . In the following, the application of the software system architecture 100 to the video data processing device 200 is taken as an example to illustrate the functions of each layer.

硬體驅動層10係用以設定各硬體元件的配置參數以控制各個硬體元件的運作，硬體元件例如為影像處理電路22及疊加電路24等。於一實施例中，硬體驅動層10將各硬體元件的配置參數設定於各硬體元件所對應的暫存器中。以疊加電路24為例，配置參數包括圖像資料於記憶體26中的物理位址、圖像資料的大小、格式、顯示位置、及透明度等，這些參數可藉由應用程式介面（Application Programming Interface，API）給上層使用。The hardware driver layer 10 is used to set the configuration parameters of each hardware component to control the operation of each hardware component, such as the image processing circuit 22 and the superposition circuit 24 . In one embodiment, the hardware driver layer 10 sets configuration parameters of each hardware element in registers corresponding to each hardware element. Taking the overlay circuit 24 as an example, the configuration parameters include the physical address of the image data in the memory 26, the size, format, display position, and transparency of the image data. , API) for the upper layer to use.

虛擬通道層20可利用虛擬的通道來設定及管理硬體的配置參數，每一個通道可對應一組配置參數，可利用多個通道來對同一硬體提供多組不同的配置參數，例如，利用多個通道來提供多組不同的配置參數給疊加電路24。在一實施例中，虛擬通道層20包含一查找表，此查找表包含各通道的通道號與所對應的配置參數的對應關係及參數資料，例如，查找表包含各通道所對應的圖像資料於記憶體26中的物理位址、圖像資料的大小、格式、顯示位置、透明度、以及視訊資料來源等資料。在一實施例中，查找表更可包含各通道與上層的幀緩衝介面及/或區域（region）模塊介面的對應關係。視訊資料處理裝置200的軟體系統可以通道號做為通道識別信號，藉以利用各通道的通道號查詢此查找表得到對應的配置參數及相關資料。The virtual channel layer 20 can use virtual channels to set and manage hardware configuration parameters. Each channel can correspond to a set of configuration parameters, and multiple channels can be used to provide multiple sets of different configuration parameters for the same hardware. For example, using Multiple channels are used to provide multiple sets of different configuration parameters to the superposition circuit 24 . In one embodiment, the virtual channel layer 20 includes a lookup table, which includes the corresponding relationship between the channel numbers of each channel and the corresponding configuration parameters and parameter data, for example, the lookup table includes image data corresponding to each channel The physical address in the memory 26, the size of the image data, the format, the display position, the transparency, and the source of the video data. In an embodiment, the lookup table may further include a correspondence relationship between each channel and an upper layer frame buffer interface and/or region module interface. The software system of the video data processing device 200 can use the channel number as a channel identification signal, so as to use the channel number of each channel to query the lookup table to obtain corresponding configuration parameters and related information.

多路視訊資料的處理原理是基於硬體處理速度足夠快的情況下，在一段時間內(例如一個幀間隔或幀週期)可以同時處理多個不同路上的視訊幀。以30fps（frames per second）的視訊資料為例，只要硬體處理一幀的資料的時間小於16毫秒，那麼它就有在33毫秒的時間內處理兩幀視訊的能力，處理速度越快，能夠處理的視訊資料路數越多。現有技術的軟體系統中，上層應用軟體係直接利用硬體驅動層10的應用程式介面，以控制疊加電路24在一路視訊資料中疊加一圖像，但是在多路視訊資料同時處理時，疊加電路24無法區分哪個視訊資料需要疊加或不疊加圖像。因此，本發明引入通道管理的概念，藉由虛擬的通道把要疊加的圖像所用的記憶體資料以及相關的設定以通道為單位進行管理，並且虛擬成專門的應用程式介面給上層使用。藉由虛擬通道層20的設置，視訊資料處理裝置200得以利用一個疊加電路24在一個幀間隔或幀週期的時間區間內對多路視訊資料進行圖像疊加處理。在一實施例中，硬體驅動層10每次收到同步訊號V _SYNC後，會根據下一張視訊幀資料中通道的標记判斷接下來要對哪個通道進行疊加處理，然後通過通道號查詢查找表以獲到該通道相關的配置參數，並在下個同步訊號V _SYNC來之前把查詢到的配置參數寫到疊加電路24的暫存器中。 The principle of multi-channel video data processing is based on the fact that the processing speed of the hardware is fast enough, and multiple video frames from different channels can be processed simultaneously within a period of time (such as a frame interval or frame period). Taking 30fps (frames per second) video data as an example, as long as the hardware processing time for one frame of data is less than 16 milliseconds, then it has the ability to process two frames of video within 33 milliseconds. The more channels of video data to process. In the software system of the prior art, the upper application software system directly utilizes the API of the hardware driver layer 10 to control the superimposing circuit 24 to superimpose an image in one channel of video data, but when multiple channels of video data are processed simultaneously, the superimposing circuit 24 It is impossible to distinguish which video material needs to be superimposed or not superimposed images. Therefore, the present invention introduces the concept of channel management, manages the memory data and related settings used by the image to be superimposed by the channel as a unit, and virtualizes it into a special application program interface for the upper layer to use. With the setting of the virtual channel layer 20 , the video data processing device 200 can use a superimposition circuit 24 to perform image superimposition processing on multiple channels of video data within a time interval of a frame interval or a frame period. In one embodiment, each time the hardware driver layer 10 receives the synchronization signal V _SYNC , it will judge which channel is to be superimposed next according to the label of the channel in the next video frame data, and then query the channel number to find Table to obtain the relevant configuration parameters of the channel, and write the configuration parameters inquired into the temporary register of the superposition circuit 24 before the next synchronous signal V _SYNC comes.

業務邏輯層30包含兩部分，一部分是標準的Linux系統的幀緩衝（Frame buffer）介面的業務邏輯，這部分會設定各個幀緩衝介面與虛擬通道層20中各通道的對應關係。另外一部分是視訊資料處理裝置200的區域(region)模塊介面相關的業務邏輯，這部分會設定各個區域模塊介面與虛擬通道層20中各通道的對應關係。舉例來說，幀緩衝介面1（/dev/fb1）可對應通道1，幀緩衝介面2（/dev/fb2）對應通道2，而區域模塊介面1（region 1）對應通道3。在一實施例中，虛擬通道層20中的查找表可包含各幀緩衝介面及各區域模塊介面與通道間的硬體驅動層10相關設定，使用者可透過特定的設定檔設定幀緩衝介面對應的虛擬通道層20通道的預設初始化狀態，而當視訊資料處理裝置200的軟體系統在初始化的過程中，會將各幀緩衝介面及各區域模塊介面與通道間的對應關係及相關設定通過虛擬通道層20的應用程式介面進行設定。The business logic layer 30 includes two parts, one part is the business logic of the frame buffer (Frame buffer) interface of the standard Linux system, and this part will set the corresponding relationship between each frame buffer interface and each channel in the virtual channel layer 20 . The other part is the business logic related to the interface of the region module of the video data processing device 200 , and this part will set the corresponding relationship between the interface of each region module and each channel in the virtual channel layer 20 . For example, framebuffer interface 1 (/dev/fb1) can correspond to channel 1, framebuffer interface 2 (/dev/fb2) can correspond to channel 2, and region module interface 1 (region 1) can correspond to channel 3. In one embodiment, the lookup table in the virtual channel layer 20 can include the hardware driver layer 10 related settings between each frame buffer interface and each area module interface and channel, and the user can configure the corresponding frame buffer interface through a specific configuration file The default initialization state of the 20 channels of the virtual channel layer, and when the software system of the video data processing device 200 is in the initialization process, the corresponding relationship and related settings between each frame buffer interface and each area module interface and channel will be passed through the virtual The API of the channel layer 20 is configured.

用戶層40用以讓使用者調用視訊資料處理裝置200中的軟體功能及/或硬體元件以實現特定應用場景。在一實施例中，用戶層40可透過幀緩衝介面在視訊資料上疊加使用者介面（User Interface，UI），用戶層40亦可透過區域模塊介面在視訊資料上疊加圖像，實現螢幕顯示（On-Screen Display，OSD）的功能。軟體系統架構100藉由設置虛擬通道層20，讓用戶層40可透過區域模塊介面以軟體分時複用多個通道在多路視訊資料疊加圖像以進行顯示。同樣的，用戶層40可透過幀緩衝介面以軟體分時複用多個通道在不同的通道上顯示不同的內容。The user layer 40 is used for allowing users to invoke software functions and/or hardware components in the video data processing device 200 to realize specific application scenarios. In one embodiment, the user layer 40 can superimpose a user interface (UI) on the video data through the frame buffer interface, and the user layer 40 can also superimpose images on the video data through the area module interface to realize screen display ( On-Screen Display, OSD) function. The software system architecture 100 allows the user layer 40 to time-multiplex multiple channels through the area module interface by setting the virtual channel layer 20 to superimpose images on multiple channels of video data for display. Similarly, the user layer 40 can use software to time-multiplex multiple channels through the frame buffer interface to display different content on different channels.

請參考圖三，圖三是根據本發明的一實施例的應用場景示意圖。以下將說明如何利用具有軟體系統架構100的視訊資料處理裝置200來實現圖三所示的應用場景。Please refer to FIG. 3, which is a schematic diagram of an application scenario according to an embodiment of the present invention. The following will describe how to use the video data processing device 200 with the software system architecture 100 to realize the application scenario shown in FIG. 3 .

請參考圖四，圖四是為實現圖三應用場景的軟體配置示意圖。如圖四所示，此系統的軟體配置包含通道1及通道2的配置參數，幀緩衝介面1（/dev/fb1）對應到通道1，幀緩衝介面2（/dev/fb2）對應到通道2，前述通道1及通道2的配置參數以及與幀緩衝介面1及幀緩衝介面2間的對應關係記錄於虛擬通道層20的查找表中。用戶層40中的圖像UI_1係使用幀緩衝介面1來進行疊加，而圖像UI_2係使用幀緩衝介面2來進行疊加。此例中，通道1用以將圖像UI_1疊加至視訊資料V _{in_1}中，通道2用以將圖像UI_2疊加至視訊資料V _{in_2}中，通道1的配置參數包含視訊資料V _{in_1}的記憶體位址及視訊大小、圖像UI_1的記憶體位址、圖像大小及格式、疊加位置（例如圖像UI_1疊加於視訊資料V _{in_1}的起始位置）、透明度等資料。通道2的配置參數包含視訊資料V _{in_2}的記憶體位址及視訊大小、圖像UI_2的記憶體位址、圖像大小及格式等資料。前述的配置參數及對應關係可在軟體系統初始化過程中透過設定檔進行配置。操作時，通道1的處理順序在通道2之前，當疊加電路驅動軟體接收到同步訊號V _{SYNC 1}時，會將對應通道1的配置參數寫入疊加電路24的暫存器中。在同步訊號V _{SYNC 1}之後，當疊加電路驅動軟體接收到同步訊號V _{SYNC 2}時，會將對應通道2的配置參數寫入疊加電路24的暫存器中，疊加電路24再據以完成疊加操作。在一實施例中，同步訊號V _{SYNC 1}及同步訊號V _{SYNC 2}可為視訊資料V _{in_1}及/或視訊資料V _{in_2}的幀同步訊號，用以指示疊加電路完成疊加一幀圖像。在一實施例中，同步訊號V _{SYNC 1}及同步訊號V _{SYNC 2}用以指示視訊資料處理裝置200完成顯示一幀圖像。 Please refer to Figure 4. Figure 4 is a schematic diagram of the software configuration for realizing the application scenario in Figure 3. As shown in Figure 4, the software configuration of this system includes the configuration parameters of channel 1 and channel 2. Frame buffer interface 1 (/dev/fb1) corresponds to channel 1, and frame buffer interface 2 (/dev/fb2) corresponds to channel 2. , the configuration parameters of the channel 1 and the channel 2 and the corresponding relationship with the frame buffer interface 1 and the frame buffer interface 2 are recorded in the lookup table of the virtual channel layer 20 . The image UI_1 in the user layer 40 uses the frame buffer interface 1 for overlaying, and the image UI_2 uses the frame buffer interface 2 for overlaying. In this example, channel 1 is used to overlay the image UI_1 into the video data V _{in_1} , and channel 2 is used to overlay the image UI_2 into the video data V _{in_2} , and the configuration parameters of channel 1 include the memory address of the video data V _{in_1} and video size, memory address of image UI_1, image size and format, overlay position (for example, image UI_1 is overlaid on the starting position of video data V _{in_1} ), transparency and other information. The configuration parameters of the channel 2 include the memory address and video size of the video data _{Vin_2} , the memory address, image size and format of the image UI_2, and the like. The aforementioned configuration parameters and corresponding relationships can be configured through configuration files during the initialization process of the software system. During operation, the processing sequence of channel 1 is before that of channel 2 , and when the superimposition circuit driver software receives the synchronous signal V _{SYNC 1} , it will write the configuration parameters corresponding to channel 1 into the register of the superposition circuit 24 . After the synchronous signal V _{SYNC 1} , when the superposition circuit driver software receives the synchronous signal V _{SYNC 2} , it will write the configuration parameters corresponding to the channel 2 into the register of the superposition circuit 24, and the superposition circuit 24 will complete the superposition operation accordingly . In one embodiment, the synchronous signal V _{SYNC 1} and the synchronous signal V _{SYNC 2} can be frame synchronous signals of the video data _{Vin_1} and/or the video data _{Vin_2} , and are used to instruct the superimposing circuit to complete superimposing a frame of images. In one embodiment, the synchronization signal V _{SYNC 1} and the synchronization signal V _{SYNC 2} are used to instruct the video data processing device 200 to finish displaying a frame of image.

在一實施例中，處理器21執行程式碼以運行具有軟體系統架構100的軟體系統。於操作O10中，藉由一影像感測器（未繪示）進行影像感測以得到感測資料S，在此實施例中，影像感測器設置於視訊資料處理裝置200的外部，並透過連接介面電性連接至視訊資料處理裝置200，而感測資料S是每秒30張而解析度為1920*1080的影像資料。於操作O20中，影像處理電路22接收感測資料S並對感測資料S進行影像處理以輸出視訊資料V _{in_1}及視訊資料V _{in_2}。在一實施例中，影像處理電路22對感測資料S所進行的影像處理可包含：去噪處理、亮度調整、色彩調整、及影像大小縮放等處理。此實施例中，視訊資料V _{in_1}的解析度為320*240，而視訊資料V _{in_2}的解析度為1920*1080，影像處理電路22依據軟體配置中配置給視訊資料V _{in_1}的記憶體位址將視訊資料V _{in_1}儲存至記憶體26，並依據配置給視訊資料V _{in_2}的記憶體位址將視訊資料V _{in_2}儲存至記憶體26。 In one embodiment, the processor 21 executes codes to run the software system having the software system architecture 100 . In operation O10, an image sensor (not shown) is used to perform image sensing to obtain sensing data S. In this embodiment, the image sensor is disposed outside the video data processing device 200 and passes through The connection interface is electrically connected to the video data processing device 200, and the sensing data S is 30 images per second with a resolution of 1920*1080. In operation O20 , the image processing circuit 22 receives the sensing data S and performs image processing on the sensing data S to output video data V _{in_1} and video data V _{in_2} . In one embodiment, the image processing performed by the image processing circuit 22 on the sensing data S may include: denoising processing, brightness adjustment, color adjustment, and image size scaling. In this embodiment, the resolution of the video data _{Vin_1} is 320*240, and the resolution of the video data _{Vin_2} is 1920*1080. The image processing circuit 22 converts the video data according to the memory address assigned to the video data _{Vin_1} in the software configuration. The data _{Vin_1} is stored in the memory 26, and the video data _{Vin_2} is stored in the memory 26 according to the memory address allocated to the video data _{Vin_2} .

於操作O30中，處理器21藉由硬體驅動層10中的疊加電路驅動軟體來控制疊加電路24，當疊加電路24或疊加電路驅動軟體於時間點T0時接收到同步訊號V _{SYNC 1}，處理器21根據虛擬通道層20的查找表，確認接下來將對通道1進行疊加處理，疊加電路驅動軟體根據查找表將對應通道1的配置參數設定至疊加電路24的暫存器中。如前面所述，通道1用以將圖像UI_1疊加至視訊資料V _{in_1}中，疊加電路24依據暫存器中的設定值自記憶體26中分別讀取視訊資料V _{in_1}及圖像UI_1並執行疊加處理1，以產生視訊資料V _{out_1}。在一實施例中，當疊加電路24或疊加電路驅動軟體接收到同步訊號V _{SYNC 2}，疊加電路24才執行將圖像UI_1疊加至視訊資料V _{in_1}中的疊加處理1，而疊加電路24花費處理時間TP1來完成疊加處理1。在一實施例中，於疊加電路24執行疊加處理1前，圖形引擎23進行繪製圖像UI_1，並將繪製好的圖像UI_1儲存至記憶體26中，圖像UI_1例如是用以顯示時間、電量參數等訊息，圖像UI_1於記憶體26中的儲存位址係對應查找表中圖像UI_1的記憶體位址。於操作O40中，視訊資料處理裝置200將視訊資料V _{out_1}輸出至一顯示器（未繪示）以進行顯示。 In operation O30, the processor 21 controls the superimposing circuit 24 through the superimposing circuit driving software in the hardware driver layer 10, and when the superimposing circuit 24 or the superimposing circuit driving software receives the synchronization signal V _{SYNC 1} at the time point T0, processing According to the lookup table of the virtual channel layer 20, the device 21 confirms that channel 1 will be overlaid next, and the overlay circuit driver software sets the configuration parameters corresponding to channel 1 into the register of the overlay circuit 24 according to the lookup table. As mentioned above, the channel 1 is used to superimpose the image UI_1 into the video data V _{in_1} , and the superimposing circuit 24 respectively reads the video data V _{in_1} and the image UI_1 from the memory 26 according to the setting value in the temporary register and executes the Overlay processing 1 to generate video data V _{out_1} . In one embodiment, when the superimposing circuit 24 or the superimposing circuit driving software receives the synchronous signal V _{SYNC 2} , the superimposing circuit 24 executes the superimposing process 1 of superimposing the image UI_1 into the video data V _{in_1} , and the superimposing circuit 24 spends processing Time TP1 to complete superposition process 1. In one embodiment, before the overlay circuit 24 performs the overlay process 1, the graphics engine 23 draws the image UI_1 and stores the drawn image UI_1 in the memory 26. The image UI_1 is used to display time, For information such as power parameters, the storage address of the image UI_1 in the memory 26 corresponds to the memory address of the image UI_1 in the lookup table. In operation O40 , the video data processing device 200 outputs the video data V _{out_1} to a display (not shown) for display.

於操作O50中，當疊加電路24或疊加電路驅動軟體於時間T1接收到同步訊號V _{SYNC 2}，處理器21根據虛擬通道層20的查找表，確認接下來將對通道2進行疊加處理2，疊加電路驅動軟體根據查找表將對應通道2的配置參數設定至疊加電路24的暫存器中。如前面所述，通道2用以將圖像UI_2疊加至視訊資料V _{in_2}中，疊加電路24依據暫存器中的設定值自記憶體26中分別讀取視訊資料V _{in_2}及圖像UI_2並執行疊加處理2，以產生視訊資料V _{out_2}。在一實施例中，於疊加電路24執行疊加處理2前，圖形引擎23進行繪製圖像UI_2，並將繪製好的圖像UI_2儲存至記憶體26中，圖像UI_2例如是用以顯示時間、日期等訊息，圖像UI_2於記憶體26中的儲存位址係對應查找表中圖像UI_2的記憶體位址。在一實施例中，同步訊號V _{SYNC 2}用以於時間點T1時觸發疊加電路24執行將圖像UI_1疊加至視訊資料V _{in_1}的疊加處理1，疊加電路24或疊加電路驅動軟體於時間點T2接收到同步訊號V _{SYNC 3}，同步訊號V _{SYNC 3}用以於時間點T2時觸發疊加電路24執行將圖像UI_2疊加至視訊資料V _{in_2}的疊加處理2，而疊加電路24花費處理時間TP2來完成疊加處理2，也就是說，疊加電路驅動軟體於疊加電路24正進行疊加處理1時，即將對應通道2的配置參數設定至疊加電路24的暫存器中。藉由這樣的機制，當疊加電路24完成將圖像UI_1疊加至視訊資料V _{in_1}的疊加處理1時，可馬上接著進行將圖像UI_2疊加至視訊資料V _{in_2}的疊加處理2，無須再等待在疊加電路24的暫存器中設定通道2的配置參數的時間。 In operation O50, when the superimposing circuit 24 or the superimposing circuit driving software receives the synchronous signal V _{SYNC 2} at time T1, the processor 21 confirms that channel 2 will be superimposed next according to the lookup table of the virtual channel layer 20, superimposing The circuit driving software sets the configuration parameters corresponding to the channel 2 into the register of the superposition circuit 24 according to the look-up table. As mentioned above, the channel 2 is used to superimpose the image UI_2 into the video data V _{in_2} , and the superimposing circuit 24 reads the video data V _{in_2} and the image UI_2 from the memory 26 according to the setting value in the temporary register and executes the Superimpose processing 2 to generate video data V _{out_2} . In one embodiment, before the overlay circuit 24 performs the overlay process 2, the graphics engine 23 draws the image UI_2 and stores the drawn image UI_2 in the memory 26. The image UI_2 is used to display time, For information such as date, the storage address of the image UI_2 in the memory 26 corresponds to the memory address of the image UI_2 in the lookup table. In one embodiment, the synchronous signal V _{SYNC 2} is used to trigger the overlay circuit 24 to execute the overlay process 1 of overlaying the image UI_1 onto the video data V _{in_1} at the time point T1, and the overlay circuit 24 or the overlay circuit driving software at the time point T2 The synchronous signal V _{SYNC 3} is received, and the synchronous signal V _{SYNC 3} is used to trigger the overlay circuit 24 to execute the overlay process 2 of overlaying the image UI_2 onto the video data V _{in_2} at the time point T2, and the overlay circuit 24 takes the processing time TP2 to complete The superimposition process 2, that is to say, the superimposition circuit driver software sets the configuration parameters corresponding to the channel 2 into the register of the superposition circuit 24 when the superposition circuit 24 is performing the superposition process 1 . With such a mechanism, when the superimposition circuit 24 completes the superimposition processing 1 of superimposing the image UI_1 on the video data V _{in_1} , it can immediately proceed to the superimposition processing 2 of superimposing the image UI_2 on the video data V _{in_2} , without waiting for the The time for setting the configuration parameters of the channel 2 is set in the temporary register of the superposition circuit 24 .

在一實施例中，視訊資料V _{in_1}及視訊資料V _{in_2}皆為每秒30張影像（30 FPS），視訊資料V _{in_1}的幀顯示週期或相鄰兩幀的間隔為33毫秒（ms），同樣的，視訊資料V _{in_2}的幀顯示週期或相鄰兩幀的間隔也是33毫秒（ms）。疊加電路24的效能需滿足處理時間TP1加處理時間TP2小於33毫秒的要求，而時間點T2與時間點T1間的時間差亦需小於33毫秒，在一較佳實施例中，時間點T2與時間點T1間的時間差小於16.5毫秒(即相鄰兩幀的間隔的一半)。本發明藉由於軟體系統中設置虛擬通道層20，通道1及通道2分別對應兩組不同的配置參數，並利用處理效能較佳的疊加電路24，透過軟體分時複用疊加電路24，實現了利用同一疊加電路來處理兩路視訊資料（如實施例中的視訊資料V _{in_1}及視訊資料V _{in_2}）的應用。 In one embodiment, both the video data _{Vin_1} and the video data _{Vin_2} are 30 images per second (30 FPS), and the frame display period of the video data _{Vin_1} or the interval between two adjacent frames is 33 milliseconds (ms). Yes, the frame display period of the video data _{Vin_2} or the interval between two adjacent frames is also 33 milliseconds (ms). The efficiency of the superposition circuit 24 needs to meet the requirement that the processing time TP1 plus the processing time TP2 is less than 33 milliseconds, and the time difference between the time point T2 and the time point T1 also needs to be less than 33 milliseconds. In a preferred embodiment, the time point T2 and time The time difference between points T1 is less than 16.5 milliseconds (that is, half of the interval between two adjacent frames). In the present invention, a virtual channel layer 20 is set in the software system, channel 1 and channel 2 respectively correspond to two sets of different configuration parameters, and the superimposition circuit 24 with better processing performance is used to multiplex the superimposition circuit 24 through software time-division multiplexing. The same superposition circuit is used to process two channels of video data (such as video data V _{in_1} and video data _{Vin_2} in the embodiment).

於操作O60中，視訊資料處理裝置200利用一編碼器（未繪示）對視訊資料V _{out_2}進行編碼處理，以產生編碼資料V _ed。編碼處理例如是依據視訊編碼標準H.264或H.265的影像編碼處理。於操作O70中，視訊資料處理裝置200利用一傳輸介面（未繪示）將編碼資料V _ed傳輸出去。在一實施例中，傳輸介面依據即時串流協定（Real Time Streaming Protocol，RTSP）將編碼資料V _ed傳輸出去。 In operation O60 , the video data processing device 200 uses an encoder (not shown) to encode the video data V _{out_2} to generate coded data V _ed . The encoding process is, for example, an image encoding process based on the video encoding standard H.264 or H.265. In operation O70, the video data processing device 200 uses a transmission interface (not shown) to transmit the encoded data V _ed . In one embodiment, the transmission interface transmits the encoded data V _ed according to Real Time Streaming Protocol (RTSP).

請參考圖五，圖五是依據本發明另一實施例所提出的一種視訊資料處理裝置的方塊示意圖。相較於圖二的視訊資料處理裝置200，視訊資料處理裝置500更包括縮放電路57。縮放電路設置於疊加電路24之前。當軟體系統架構100應用於視訊資料處理裝置500的場景時，虛擬通道層20除了利用多個通道來提供多組不同的配置參數給疊加電路24外，虛擬通道層20更利用該些通道來提供多組不同的配置參數給縮放電路57。在一實施例中，虛擬通道層20中的查找表包含各通道的通道號與縮放電路57所對應的配置參數的對應關係及參數資料，以及各通道的通道號與疊加電路24所對應的配置參數的對應關係及參數資料。舉例來說，圖四的軟體配置中查找表更可包括通道1對應縮放電路57的配置參數，用以控制縮放電路57對視訊資料V _{in_1}進行縮放處理，查找表也包括通道2對應縮放電路57的配置參數，用以控制縮放電路57對視訊資料V _{in_2}進行縮放處理。操作時，當硬體驅動層10接收到同步訊號V _{SYNC 1}時，會將對應通道1的配置參數寫入縮放電路57及疊加電路24各自的暫存器中。縮放電路57據以對視訊資料V _{in_1}進行縮放處理後輸出經處理後的視訊資料至疊加電路24，以進行疊加處理。在同步訊號V _{SYNC 1}之後，當硬體驅動層10接收到同步訊號V _{SYNC 2}時，會將對應通道2的配置參數寫入縮放電路57及疊加電路24各自的暫存器中。縮放電路57據以對視訊資料V _{in_2}進行縮放處理後輸出經處理後的視訊資料至疊加電路24，以進行疊加處理。縮放電路57的效能需滿足在相鄰兩幀的間隔時間內完成對視訊資料V _{in_1}及視訊資料V _{in_2}的縮放處理。在一較佳實施例中，縮放電路57完成對視訊資料V _{in_1}及視訊資料V _{in_2}的縮放處理所需的時間加總疊加電路24完成疊加處理1及疊加處理2所需的時間係小於相鄰兩幀的間隔時間（此例中為33毫秒）。 Please refer to FIG. 5 . FIG. 5 is a schematic block diagram of a video data processing device according to another embodiment of the present invention. Compared with the video data processing device 200 in FIG. 2 , the video data processing device 500 further includes a scaling circuit 57 . The scaling circuit is arranged before the superposition circuit 24 . When the software system architecture 100 is applied to the scene of the video data processing device 500, the virtual channel layer 20 not only uses multiple channels to provide multiple sets of different configuration parameters to the overlay circuit 24, but also uses these channels to provide Multiple sets of different configuration parameters are given to the scaling circuit 57 . In one embodiment, the lookup table in the virtual channel layer 20 includes the corresponding relationship and parameter data between the channel number of each channel and the configuration parameter corresponding to the scaling circuit 57, as well as the channel number of each channel and the configuration corresponding to the superposition circuit 24 Parameter correspondence and parameter data. For example, the look-up table in the software configuration of FIG. 4 may further include the configuration parameters of the scaling circuit 57 corresponding to channel 1, so as to control the scaling circuit 57 to perform scaling processing on the video data V _{in_1} , and the look-up table also includes the scaling circuit 57 corresponding to channel 2. The configuration parameters are used to control the scaling circuit 57 to perform scaling processing on the video data V _{in_2} . During operation, when the hardware driver layer 10 receives the synchronous signal V _{SYNC 1} , it will write the configuration parameters corresponding to the channel 1 into respective registers of the scaling circuit 57 and the superposition circuit 24 . The scaling circuit 57 performs scaling processing on the video data V _{in_1} and then outputs the processed video data to the overlay circuit 24 for overlay processing. After the synchronization signal V _{SYNC 1} , when the hardware driver layer 10 receives the synchronization signal V _{SYNC 2} , it will write the configuration parameters corresponding to the channel 2 into respective registers of the scaling circuit 57 and the superposition circuit 24 . The scaling circuit 57 scales the video data V _{in_2} accordingly and then outputs the processed video data to the overlay circuit 24 for overlay processing. The performance of the scaling circuit 57 needs to be able to complete the scaling process on the video data V _{in_1} and the video data V _{in_2} within the interval between two adjacent frames. In a preferred embodiment, the time required for the scaling circuit 57 to complete the scaling process of the video data _{Vin_1} and the video data _{Vin_2} is shorter than the time required for the superposition circuit 24 to complete the superimposition process 1 and the superimposition process 2. The time between two frames (33 milliseconds in this example).

圖三及圖四的實施例中說明了本發明的軟體系統架構及視訊資料處理裝置可透過軟體分時複用同一疊加電路來處理兩路視訊資料，在硬體效能條件許可下，本發明的軟體系統架構及視訊資料處理裝置更可透過軟體分時複用同一硬體電路來處理兩路以上的視訊資料。圖三及圖四的實施例中是以通道1及通道2皆對應到幀緩衝介面為例進行說明，實施上，各通道皆可選擇對應幀緩衝介面或區域模塊介面。The embodiment in Fig. 3 and Fig. 4 illustrates that the software system architecture and video data processing device of the present invention can process two channels of video data through software time-division multiplexing with the same superposition circuit. Under the condition of hardware performance, the present invention The software system architecture and video data processing device can process more than two channels of video data by time-division multiplexing the same hardware circuit through software. In the embodiments of FIG. 3 and FIG. 4 , the channel 1 and the channel 2 are both corresponding to the frame buffer interface for illustration. In practice, each channel can be selected to correspond to the frame buffer interface or the area module interface.

綜上所述，本發明的軟體系統架構及視訊資料處理裝置藉由在軟體架構中設置虛擬通道層20，以實現透過軟體分時複用硬體電路在多路視訊資料上疊加不同的緩衝介面及/或區域模塊介面所對應的内容，大幅提昇硬體的使用率，並達到多個視窗顯示OSD/UI的效果。To sum up, the software system architecture and video data processing device of the present invention set the virtual channel layer 20 in the software architecture to realize the superimposition of different buffer interfaces on multiple video data through software time-division multiplexing hardware circuits And/or the content corresponding to the interface of the area module, which greatly improves the utilization rate of the hardware, and achieves the effect of displaying OSD/UI in multiple windows.

雖然本發明之實施例如上所述，然而該些實施例並非用來限定本發明，本技術領域具有通常知識者可依據本發明之明示或隱含之內容對本發明之技術特徵施以變化，凡此種種變化均可能屬於本發明所尋求之專利保護範疇，換言之，本發明之專利保護範圍須視本說明書之申請專利範圍所界定者為準。Although the embodiments of the present invention are as described above, these embodiments are not intended to limit the present invention, and those skilled in the art can make changes to the technical characteristics of the present invention according to the explicit or implicit contents of the present invention. All these changes may belong to the scope of patent protection sought by the present invention. In other words, the scope of patent protection of the present invention must be defined by the scope of patent application in this specification.

100 軟體系統架構 200、500 視訊資料處理裝置 10 硬體驅動層 20 虛擬通道層 21 處理器 22 影像處理電路 23 圖形引擎 24 疊加電路 25 記憶體界面 26 記憶體 30 業務邏輯層 40 用戶層 57 縮放電路 O10、O20、O30、O40、O50、O60、O70 操作 100 Software System Architecture 200, 500 video data processing device 10 hard drive layer 20 virtual channel layer 21 processors 22 Image processing circuit 23 graphics engine 24 superposition circuit 25 memory interface 26 memory 30 business logic layer 40 user level 57 scaling circuit O10, O20, O30, O40, O50, O60, O70 Operation

為了更清楚地說明本發明實施例中的技術方案，下面將對實施例描述中所需要使用的附圖作簡單地介紹，顯而易見地，下面描述中的附圖僅僅是本發明的一些實施例，對於本領域技術人員來講，在不付出創造性勞動的前提下，還可以根據這些附圖獲得其他的實施例。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other embodiments can also be obtained according to these drawings without any creative effort.

圖一是依據本發明一實施例所提出的一種軟體系統架構的示意圖；FIG. 1 is a schematic diagram of a software system architecture proposed according to an embodiment of the present invention;

圖二是依據本發明一實施例所提出的一種視訊資料處理裝置的方塊示意圖；FIG. 2 is a schematic block diagram of a video data processing device according to an embodiment of the present invention;

圖三是根據本發明的一實施例的應用場景示意圖；FIG. 3 is a schematic diagram of an application scenario according to an embodiment of the present invention;

圖四是為實現圖三應用場景的軟體配置示意圖；Figure 4 is a schematic diagram of software configuration for realizing the application scenario in Figure 3;

圖五是依據本發明另一實施例所提出的一種視訊資料處理裝置的方塊示意圖。FIG. 5 is a schematic block diagram of a video data processing device according to another embodiment of the present invention.

Claims

一種視訊資料處理方法，包括：提供一查找表，該查找表包含一第一通道識別信號與一第一組參數的對應關係、及一第二通道識別信號與一第二組參數的對應關係；根據該第一通道識別信號查詢該查找表，以將該第一組參數寫入一疊加電路的暫存器；利用該疊加電路，根據該第一組參數於一第一時間點將一第一圖像疊加於一第一視訊資料中；根據該第二通道識別信號查詢該查找表，以將該第二組參數寫入該疊加電路的暫存器；以及利用該疊加電路，根據該第二組參數於一第二時間點將一第二圖像疊加於一第二視訊資料中；其中，該第一時間點與該第二時間點間的時間差小於該第一視訊資料的一幀間隔。 A video data processing method, comprising: A lookup table is provided, the lookup table includes a correspondence between a first channel identification signal and a first group of parameters, and a correspondence between a second channel identification signal and a second group of parameters; Querying the look-up table according to the first channel identification signal, so as to write the first set of parameters into a register of a superposition circuit; using the superimposing circuit to superimpose a first image into a first video data at a first time point according to the first set of parameters; querying the look-up table according to the second channel identification signal, so as to write the second set of parameters into the register of the superposition circuit; and using the superimposing circuit to superimpose a second image on a second video data at a second time point according to the second set of parameters; Wherein, the time difference between the first time point and the second time point is less than a frame interval of the first video data.

如請求項1之視訊資料處理方法，其中，該疊加電路將該第一圖像疊加於該第一視訊資料中花費一第一處理時間，該疊加電路將該第二圖像疊加於該第二視訊資料中花費一第二處理時間，其中，該第一處理時間與該第二處理時間的總合小於該幀間隔。The video data processing method according to claim 1, wherein the superimposing circuit spends a first processing time on superimposing the first image on the first video data, and the superimposing circuit superimposes the second image on the second A second processing time is spent in the video data, wherein the sum of the first processing time and the second processing time is less than the frame interval.

如請求項1之視訊資料處理方法，其中，該第二組參數係於該疊加電路正進行將該第一圖像疊加於該第一視訊資料的操作中寫入該疊加電路的暫存器。The video data processing method according to claim 1, wherein the second set of parameters is written into the register of the superimposing circuit when the superimposing circuit is superimposing the first image on the first video data.

如請求項1之視訊資料處理方法，其中，該查找表更包含該第一通道識別信號與Linux***中一第一幀緩衝介面的對應關係。The video data processing method according to claim 1, wherein the lookup table further includes a corresponding relationship between the first channel identification signal and a first frame buffer interface in the Linux system.

如請求項1之視訊資料處理方法，其中，該查找表設置於一軟體的虛擬通道層中，該虛擬通道層係設置於該軟體的一業務邏輯層及一硬件驅動層之間。The video data processing method of Claim 1, wherein the lookup table is set in a virtual channel layer of a software, and the virtual channel layer is set between a business logic layer and a hardware driver layer of the software.

如請求項1之視訊資料處理方法，其中，該第一組參數包含該第一圖像於一內存的位置、該第一圖像的大小、及該第一圖像疊加於該第一視訊資料的起始位置。The video data processing method according to claim 1, wherein the first set of parameters includes the location of the first image in a memory, the size of the first image, and the superposition of the first image on the first video data the starting position of .

如請求項1之視訊資料處理方法，其中，該第一時間點對應於一第一幀同步信號，該第二時間點對應於一第二幀同步信號。The video data processing method according to claim 1, wherein the first time point corresponds to a first frame synchronization signal, and the second time point corresponds to a second frame synchronization signal.

一種視訊資料處理裝置，其包括：一處理器，根據一第一通道識別信號提供一第一組參數，並根據一第二通道識別信號提供一第二組參數；以及一疊加電路，根據該第一組參數於一第一時間點將一第一圖像疊加於一第一視訊資料中，且根據該第二組參數於一第二時間點將一第二圖像疊加於一第二視訊資料中；其中，該第一時間點與該第二時間點間的時間差小於該第一視訊資料的一幀間隔。 A video data processing device, comprising: a processor, providing a first set of parameters according to a first channel identification signal, and providing a second set of parameters according to a second channel identification signal; and A superposition circuit, superimposing a first image on a first video data at a first time point according to the first set of parameters, and superimposing a second image on a second time point according to the second set of parameters - In the second video data; Wherein, the time difference between the first time point and the second time point is less than a frame interval of the first video data.

如請求項8之視訊資料處理裝置，其中，該疊加電路將該第一圖像疊加於該第一視訊資料中花費一第一處理時間，該疊加電路將該第二圖像疊加於該第二視訊資料中花費一第二處理時間，其中，該第一處理時間與該第二處理時間的總合小於該幀間隔。The video data processing device according to claim 8, wherein the superimposing circuit spends a first processing time on superimposing the first image on the first video data, and the superimposing circuit superimposes the second image on the second A second processing time is spent in the video data, wherein the sum of the first processing time and the second processing time is less than the frame interval.

如請求項8之視訊資料處理裝置，其中，該處理器於該第二時間點之前將該第二組參數寫入用以控制該疊加電路的暫存器。The video data processing device according to claim 8, wherein the processor writes the second set of parameters into a register for controlling the superimposition circuit before the second time point.

如請求項8之視訊資料處理裝置，其中，該第一通道識別信號對應Linux***中一第一幀緩衝介面，該第二通道識別信號對應該Linux***中一第二幀緩衝介面。The video data processing device according to claim 8, wherein the first channel identification signal corresponds to a first frame buffer interface in the Linux system, and the second channel identification signal corresponds to a second frame buffer interface in the Linux system.

如請求項11之視訊資料處理裝置，其中，該處理器根據該第一通道識別信號查詢一查找表，以得到該第一組參數。The video data processing device according to claim 11, wherein the processor queries a look-up table according to the first channel identification signal to obtain the first set of parameters.

如請求項12之視訊資料處理裝置，其中，該查找表包含該第一通道識別信號與與該第一幀緩衝介面及該第一組參數的對應關係。The video data processing device according to claim 12, wherein the lookup table includes the correspondence between the first channel identification signal and the first frame buffer interface and the first set of parameters.

如請求項13之視訊資料處理裝置，其中，該第一組參數包含該第一圖像於一內存的位置、該第一圖像的大小、及該第一圖像疊加於該第一視訊資料的起始位置。The video data processing device according to claim 13, wherein the first set of parameters includes the location of the first image in a memory, the size of the first image, and the superposition of the first image on the first video data the starting position of .

如請求項8之視訊資料處理裝置，其中，該第一通道識別信號對應Linux***中一幀緩衝介面，該第二通道識別信號對應一區域模塊介面。The video data processing device according to claim 8, wherein the first channel identification signal corresponds to a frame buffer interface in the Linux system, and the second channel identification signal corresponds to a local module interface.

如請求項8之視訊資料處理裝置，其中，該第一時間點對應於一第一幀同步信號，該第二時間點對應於一第二幀同步信號。The video data processing device according to claim 8, wherein the first time point corresponds to a first frame synchronization signal, and the second time point corresponds to a second frame synchronization signal.

如請求項8之視訊資料處理裝置，更包括：一縮放電路，對該第一視訊資料進行縮放處理，以將處理後的該第一視訊資料輸出至該疊加電路；其中，該第一組參數包含控制該縮放電路的參數。 Such as the video data processing device of claim item 8, further comprising: a scaling circuit, performing scaling processing on the first video data, so as to output the processed first video data to the superposition circuit; Wherein, the first group of parameters includes parameters for controlling the scaling circuit.