CN114189712B - Virtual VDP device supporting decoding frame rate control in video monitoring and application - Google Patents
Virtual VDP device supporting decoding frame rate control in video monitoring and application Download PDFInfo
- Publication number
- CN114189712B CN114189712B CN202111508127.0A CN202111508127A CN114189712B CN 114189712 B CN114189712 B CN 114189712B CN 202111508127 A CN202111508127 A CN 202111508127A CN 114189712 B CN114189712 B CN 114189712B
- Authority
- CN
- China
- Prior art keywords
- module
- frame rate
- video
- vpss
- virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 12
- 239000000872 buffer Substances 0.000 claims abstract description 109
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000013144 data compression Methods 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 7
- 230000006870 function Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000010365 information processing Effects 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 241001522296 Erithacus rubecula Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a virtual VDP device supporting decoding frame rate control in video monitoring and application thereof, and relates to the technical field of video processing. The device comprises a VDH module, a VPSS module and a virtual VDP module; the VDH module decodes the video code stream according to the analysis result and configuration of the video decoding firmware and generates decoding data; the VPSS module receives the decoded data, processes the video and transmits the processed video to the virtual VDP module; the virtual VDP module creates a private video cache pool, performs buffer rotation with the VPSS module, and controls the frame rate of the whole channel; the virtual VDP module is provided with a frame rate control unit which is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module. The invention can obviously reduce the memory requirement and the hardware requirement of the system in the non-display scene.
Description
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a virtual VDP device supporting decoding frame rate control in video monitoring and an application thereof.
Background
In the video surveillance industry, the video decoder consists of video decoding firmware VFMW (video firmware) running on an ARM processor and an embedded hardware video decoding engine VDH (video decoding module for high definition). The video decoding firmware VFMW obtains a code stream from upper software, analyzes the code stream and calls VDH to generate a decoded image sequence. The decoded data stream is bound to the video processing subsystem VPSS (video processing sub system) under the control of the upper layer software and sent to the video processing subsystem VPSS, which is then output to a display or other device via the video display processor VDP (video display processor) module. As an example of a typical manner, fig. 1 illustrates a standard display flow of a surveillance video image: (1) First, the VDP creates a Private video cache pool (i.e., private VB pool, full name Private video buffer pool) for image display. (2) The VDH calls for retrieving Common VB blocks (i.e., common cache blocks) from a Common video cache pool (i.e., common VB pool, collectively Common video buffer pool). (3) And after the VDH decoding processing is finished, the public VB blocks are transmitted to the VPSS by the buffer, and the VPSS occupies the transmitted public VB blocks. (4) The VPSS obtains the private VB block (i.e., private cache block) from the VDP and occupies the incoming private VB block. (5) After the VPSS processes the private VB block transmitted in the step (4), a buffer is transmitted to the VDP, and the VDP occupies the transmitted private VB block. (6) The VPSS calls a user sub-module user_sub to release the public VB blocks transmitted by the VDH in the third step. (7) Data is output to a display through the VDP to display an image. The video display processor VDP generally allocates the size of the off-chip frame buffer space of each frame according to the actual resolution of the display video, and allocates the frame number of the frame buffer according to the buffer queue depth actually required, so that the buffer queue allocation can be adaptive to a certain extent.
In the current security monitoring system, when a VDP channel (or path) is created, a constant frame buffer size is generally allocated to each channel. For some special scenes which do not need to be displayed, for example, some scenes only need to be decoded normally, after the data is sent to the VPSS round, the data is not needed to be sent to the VDP module to be displayed, a standard display process is created by adopting the process, and a constant frame buffer size is allocated to each channel. The scheme increases the waste of hardware VDP equipment on one hand and causes the resource shortage of the frame cache memory and the large pressure of bandwidth on the other hand. Taking as an example to create a VDP channel, if a YUV422 output format is adopted, assuming that three round robin buffers (buffers) are set, according to the above conventional display procedure, the VDP needs to create a private video buffer (buffer) pool according to the size of the actual display, for example, the display resolution of 4k@30 needs to create 3840×2160x2x3, which approximately needs 47M of memory. If multiple channels are created, for an application scenario where the input video resolution and number determine to be multiplexed, the total buffer space size of the frames required for existing display channel creation is substantially fixed, and the size of the buffer space is equal to the number of buffers per frame of the display resolution. If 4 paths (VDH-VPSS-VDP channels) are independently running, the system needs to allocate a total of about 188M (i.e., 47m×4) of memory for the VDP module according to 3 4K display buffers per path. With the increase of display resolution and the number of devices, the memory and bandwidth of the video display unit also increase significantly, the size supported in the latest ultra-high definition display has reached 8K, if VDP hardware resources are still used, about 752M of memory is required to be allocated (the display resolution is changed to 8K for display), and the off-chip memory cost required for creating the cache of the display is very high.
In summary, how to optimize hardware resources and frame buffer sizes to reduce hardware resource waste and reduce system bandwidth on the basis of ensuring normal decoding of VDH for application scenarios that require normal decoding without transmission is a technical problem that needs to be solved currently.
Disclosure of Invention
The invention aims at: the virtual VDP device and the application supporting the decoding frame rate control in video monitoring are provided to overcome the defects of the prior art. The virtual VDP device combines the virtual equipment and the frame rate control, the hardware video decoding engine VDH module is bound with the video processing subsystem VPSS module, the VPSS module is bound to the virtual VDP module based on software, the virtual VDP module and the VPSS module are used for cache interaction, and the frame rate of the whole channel is controlled.
In order to achieve the above object, the present invention provides the following technical solutions:
a virtual VDP device supporting decoding frame rate control in video monitoring comprises a VDH module, a VPSS module and a virtual VDP module;
the VDH module is used for decoding the video code stream according to the analysis result and configuration of the video decoding firmware, generating decoded data and transmitting the decoded data to the VPSS module;
the VPSS module is used for receiving the decoded data, performing video processing and transmitting the processed video to the virtual VDP module;
the virtual VDP module is used for creating a private video cache pool, performing buffer rotation with the VPSS module and controlling the frame rate of the whole channel; the frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
Further, the manner in which the virtual VDP module and the VPSS module perform buffer rotation is as follows:
the VDH module acquires a public VB block from the public video cache pool to perform decoding processing, and transmits a buffer to the VPSS module after the public VB block is decoded; the VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed;
the virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
Further, the frame rate control unit is disposed within the virtual VDP module and connected to the VPSS module, and the frame rate control unit is configured to: configuring the speed of a virtual VDP module for acquiring a private VB block from a private video cache pool so as to control the buffer processing rate of the virtual VDP module;
according to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to be matched with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
Further, a mode switching unit is further arranged corresponding to the virtual VDP module, and the mode switching unit is used for switching the virtual VDP module between a non-frame rate control mode and a frame rate control mode;
in a non-frame rate control mode, closing the frame rate control unit, wherein the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the time interval of a preset timer, so as to control the decoding frame rate of the VDH module;
in the frame rate control mode, the frame rate control unit is started, and the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the frame rate configured by the application layer, so as to control the decoding frame rate of the VDH.
Further, in the frame rate control mode, the frame rate control unit is configured to: and acquiring frame rate information configured by an API interface of an application layer, and controlling the buffer processing rate of the virtual VDP module according to the configured frame rate information.
Further, the VDH module and video decoding firmware constitute a video decoder that supports video decoding for multiple video formats.
Further, the VPSS module is configured to perform one or more of video cropping, setting frame size and color configuration, blocking processing, video occlusion, video overlay, and/or video data compression on the decoded data.
The invention also provides a decoding frame rate control method without sending display in video monitoring, which comprises the following steps:
according to the analysis result and the configuration information of the video decoding firmware, the VDH module decodes the video code stream, generates decoding data and transmits the decoding data to the VPSS module;
the VPSS module receives the decoded data and performs video processing, the processed video is transmitted to the virtual VDP module, and the virtual VDP module is used for creating a private video buffer pool, performing buffer rotation with the VPSS module and controlling the frame rate of the whole channel; the frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
Further, the steps of buffer rotation performed by the virtual VDP module and the VPSS module are as follows:
the VDH module acquires a public VB block from the public video cache pool to perform decoding processing, and transmits a buffer to the VPSS module after the public VB block is decoded; the VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed;
the virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
Further, the frame rate control unit is arranged in the virtual VDP module and connected with the VPSS module, and the frame rate control unit controls the buffer processing rate of the virtual VDP module by configuring the speed of the virtual VDP module for acquiring the private VB block from the private video cache pool;
according to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to match with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
Compared with the prior art, the invention has the following advantages and positive effects by taking the technical scheme as an example: the virtual VDP device combines virtual equipment and frame rate control, the hardware video decoding engine VDH module is bound with the video processing subsystem VPSS module, the VPSS module is bound to the virtual VDP module based on software, cache interaction is carried out between the virtual VDP module and the VPSS module, and the frame rate of the whole channel is controlled.
Drawings
Fig. 1 is a flow chart of information processing for decoding and displaying a code stream in the prior art.
Fig. 2 is a flowchart of information processing of a virtual VDP coding apparatus supporting decoding frame rate control according to an embodiment of the present invention.
Detailed Description
The following describes in further detail a virtual VDP device and application supporting decoding frame rate control in video surveillance disclosed in the present invention with reference to the accompanying drawings and specific embodiments. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be regarded as being isolated, and they may be combined with each other to achieve a better technical effect. In the drawings of the embodiments described below, like reference numerals appearing in the various drawings represent like features or components and are applicable to the various embodiments. Thus, once an item is defined in one drawing, no further discussion thereof is required in subsequent drawings.
It should be noted that the structures, proportions, sizes, etc. shown in the drawings are merely used in conjunction with the disclosure of the present specification, and are not intended to limit the applicable scope of the present invention, but rather to limit the scope of the present invention. The scope of the preferred embodiments of the present invention includes additional implementations in which functions may be performed out of the order described or discussed, including in a substantially simultaneous manner or in an order that is reverse, depending on the function involved, as would be understood by those of skill in the art to which embodiments of the present invention pertain.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of the exemplary embodiments may have different values.
Examples
Referring to fig. 2, a virtual VDP device supporting decoding frame rate control in video monitoring is provided in this embodiment.
The virtual VDP device may specifically include a VDH module, a VPSS module, and a virtual VDP module.
The VDH (video decoding module for high definition, i.e., hardware video decoding engine) module is configured to decode the video bitstream and generate decoded data according to the parsing result and configuration of the video decoding firmware VFMW (Video Firmware), and transmit the decoded data to the VPSS module.
The VDH module and video decoding firmware VFMW constitute a video decoder that supports video decoding for multiple video formats. The video decoding firmware VFMW is operated on the ARM processor and is used for obtaining a code stream from upper software, analyzing the code stream, calling the VDH module to decode the code stream, generating a decoded image sequence and forming decoded data.
By way of example and not limitation, the video decoder may support decoding of various video formats such as h.264, h.265, AVS, etc.
The VPSS (video processing sub system, i.e., video processing subsystem) module is configured to receive the decoded data and perform video processing, and transmit the processed video to the virtual VDP module.
The VPSS module is used for realizing the video processing function. In this embodiment, the VPSS module supports video clipping, setting frame size and color configuration, block processing, video occlusion, video overlay, video data compression, and other video processing functions.
The virtual VDP (video display processor, i.e., video display processor) module is configured to create a Private video buffer pool (Private VB pool), perform buffer rotation with the VPSS module, and control the frame rate of the entire path. The frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
In contrast to conventional hardware VDP, the virtual VDP module can also actively read video and graphics data from the corresponding locations in memory, except that it does not have to be sent to a display for display.
In this embodiment, the specific manner in which the virtual VDP module and the VPSS module perform buffer rotation may be as follows: the VDH module obtains a Common VB block from a Common video cache pool (Common VB pool) to carry out decoding processing, and transmits a buffer to the VPSS module after the Common VB block is decoded. The VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed. The virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
The above process is described in detail in connection with fig. 2.
Step 1, the virtual VDP module creates a Private VB pool (Private VB pool) for interaction with buffers of the VPSS module and frame rate control of the whole path.
And 2, the VDH module calls a Common VB pool (Common VB pool) to acquire a Common VB block from the Common VB pool.
And 3, the VDH module decodes and processes the public VB blocks and transmits buffers to the VPSS, and the VPSS occupies the transmitted public VB blocks.
And 4, the VPSS module acquires the private VB blocks transmitted from the virtual VDP module and occupies the transmitted private VB blocks.
And 5, after the VPSS module processes the private VB blocks from the step 4, transmitting buffers to the virtual VDP module, wherein the virtual VDP module occupies the transmitted private VB blocks.
And 6, the VPSS module calls a user sub module to release the public VB blocks transmitted by the VDH module in the step 3.
Preferably, the frame rate control unit is disposed in the virtual VDP module and connected to the VPSS module. For a VDH-VPSS-virtual VDP channel (or path), the decoding frame rate allowed by the decoder is often significantly greater than the frame rate processing speed allowed by the VDP, and the decoding frame rate needs to be controlled to ensure that the buffer of the whole channel is guaranteed to rotate normally. In this embodiment, the frame rate control unit configures the speed of the virtual VDP module to acquire the private VB block from the private video buffer pool to control the buffer processing rate of the virtual VDP module, thereby controlling the decoding frame rate of the VDH module.
Specifically, the frame rate control unit is configured to: and configuring the speed of acquiring the private VB block from the private video cache pool by the virtual VDP module to control the buffer processing rate of the virtual VDP module. According to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to be matched with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
By way of example, but not limitation, for example, the speed at which the configured virtual VDP module obtains the private VB block from the private video cache pool is M, and the private buffer processing rate of the virtual VDP module is M. Because the number of private buffer buffers is limited, the frame processing capacity of the VPSS module is often much greater than that of the VDP module, that is, the processing speed of the VDP module in the channel determines the actual processing speed of the VPSS module, and the actual processing speed of the VPSS module is limited by the VDP module, and it is assumed that the processing frame rate corresponding to the speed M is 60 frames per second, so that the buffer rate taken by the VPSS module is also 60 frames per second. Also because the frame processing capacity of the VDH module is often greater than that of the VPSS module, that is, the actual processing speed of the VDH module in the channel is limited by the VPSS module, the buffer rate taken by the VPSS module is 60 frames per second, resulting in the processing speed of the VDH module also being 60 frames per second. The processing frame rate of the whole channel can be controlled by configuring the buffer processing rate of the virtual VDP module, so that the aim of controlling the decoding frame rate of the VDH module is fulfilled.
In another implementation manner of this embodiment, a mode switching unit is further provided corresponding to the virtual VDP module, where the mode switching unit is configured to switch the virtual VDP module between a non-frame rate control mode and a frame rate control mode.
And in the non-frame rate control mode, closing the frame rate control unit, and controlling the buffer processing rate of the virtual VDP module by the virtual VDP module according to the time interval of a preset timer so as to control the decoding frame rate of the VDH module.
In the frame rate control mode, the frame rate control unit is started, and the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the frame rate configured by the application layer, so as to control the decoding frame rate of the VDH.
In a frame rate control mode, the frame rate control unit is configured to: and acquiring frame rate information configured by an API interface of an application layer, and controlling the buffer processing rate of the virtual VDP module according to the configured frame rate information.
In the above embodiment, the virtual VDP module may be divided into two modes, and a user may configure the modes of operation of the virtual VDP module through an API interface of an upper layer. The non-frame rate control mode is suitable for the situation without frame rate control requirement, in which the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the interval of the timer of the upper layer software, and further controls the frame rate of the virtual VDP module, so that the VDH module decodes according to the frame rate of the virtual VDP module. The frame rate control mode is suitable for a case where frame rate control is required, in which the virtual VDP controls the decoding frame rate of the VDH according to the frame rate configured by the application layer, and specifically, may be controlled by the frame rate control unit according to the API interface configuration frame rate information of the upper layer.
The virtual VDP device provided by the invention has the advantages that all the API interface information is configured to be the same as the interface information of the conventional hardware VDP device, so that all the calling interfaces of the application layer are unified, and the operation of the virtual VDP device is controlled under the condition that the display is not required. Compared with the conventional hardware VDP scheme, the method does not need to create the buffer according to the display resolution, can configure the size of the buffer according to the user-defined information of the user, remarkably reduces the system cost, can save about 99% of system memory by calculating according to the 8k display resolution, remarkably saves hardware resources and reduces the system bandwidth.
The invention also provides a decoding frame rate control method without sending display in video monitoring. The method comprises the following steps:
and S100, decoding the video code stream by the VDH module according to the analysis result and the configuration information of the video decoding firmware, generating decoded data, and transmitting the decoded data to the VPSS module.
S200, the VPSS module receives decoded data and performs video processing, the processed video is transmitted to the virtual VDP module, and the virtual VDP module is used for creating a private video buffer pool, performing buffer rotation with the VPSS module and controlling the frame rate of the whole channel; the frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
In this embodiment, specifically, the steps of performing buffer rotation by the virtual VDP module and the VPSS module are as follows: the VDH module acquires a public VB block from the public video cache pool to perform decoding processing, and transmits a buffer to the VPSS module after the public VB block is decoded; the VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed. The virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
In this embodiment, the frame rate control unit is disposed in the virtual VDP module and is connected to the VPSS module, and the frame rate control unit controls the buffer processing rate of the virtual VDP module by configuring the speed at which the virtual VDP module obtains the private VB block from the private video cache pool. According to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to match with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
Other technical features are referred to the previous embodiments and will not be described here again.
The scheme is suitable for the scene without display under the VDH- > VPSS- > VDP channel, can remarkably reduce the memory allocation requirement, reduce the hardware requirement, save the resources and reduce the system bandwidth.
In the above description, the disclosure of the present invention is not intended to limit itself to these aspects. Rather, the components may be selectively and operatively combined in any number within the scope of the present disclosure. In addition, terms like "comprising," "including," and "having" should be construed by default as inclusive or open-ended, rather than exclusive or closed-ended, unless expressly defined to the contrary. All technical, scientific, or other terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Common terms found in dictionaries should not be too idealized or too unrealistically interpreted in the context of the relevant technical document unless the present disclosure explicitly defines them as such. Any alterations and modifications of the present invention, which are made by those of ordinary skill in the art based on the above disclosure, are intended to be within the scope of the appended claims.
Claims (10)
1. The virtual VDP device supporting decoding frame rate control in video monitoring is characterized by comprising a VDH module, a VPSS module and a virtual VDP module;
the VDH module is used for decoding the video code stream according to the analysis result and configuration of the video decoding firmware, generating decoded data and transmitting the decoded data to the VPSS module;
the VPSS module is used for receiving the decoded data, performing video processing and transmitting the processed video to the virtual VDP module;
the virtual VDP module is used for creating a private video cache pool, performing buffer rotation with the VPSS module and controlling the frame rate of the whole channel; the frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
2. The virtual VDP device of claim 1, wherein the virtual VDP module and the VPSS module perform buffer rotation in the following manner:
the VDH module acquires a public VB block from the public video cache pool to perform decoding processing, and transmits a buffer to the VPSS module after the public VB block is decoded; the VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed;
the virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
3. The virtual VDP apparatus according to claim 1 or 2, wherein the frame rate control unit is provided within the virtual VDP module and connected to the VPSS module, the frame rate control unit being configured to: configuring the speed of a virtual VDP module for acquiring a private VB block from a private video cache pool so as to control the buffer processing rate of the virtual VDP module;
according to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to be matched with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
4. The virtual VDP apparatus of claim 3, further comprising a mode switching unit corresponding to the virtual VDP module, the mode switching unit configured to switch the virtual VDP module between a non-frame rate control mode and a frame rate control mode;
in a non-frame rate control mode, closing the frame rate control unit, wherein the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the time interval of a preset timer, so as to control the decoding frame rate of the VDH module;
in the frame rate control mode, the frame rate control unit is started, and the virtual VDP module controls the buffer processing rate of the virtual VDP module according to the frame rate configured by the application layer, so as to control the decoding frame rate of the VDH.
5. The virtual VDP apparatus of claim 4, wherein in a frame rate control mode, the frame rate control unit is configured to: and acquiring frame rate information configured by an API interface of an application layer, and controlling the buffer processing rate of the virtual VDP module according to the configured frame rate information.
6. The virtual VDP apparatus of claim 1, wherein the VDH module and video decoding firmware constitute a video decoder that supports video decoding in a plurality of video formats.
7. The virtual VDP apparatus of claim 1, wherein the VPSS module is configured to perform one or more of video cropping, frame size and color configuration setting, blocking processing, video occlusion, video overlay, and/or video data compression on the decoded data.
8. A decoding frame rate control method without sending display in video monitoring is characterized by comprising the following steps:
according to the analysis result and the configuration information of the video decoding firmware, the VDH module decodes the video code stream, generates decoding data and transmits the decoding data to the VPSS module;
the VPSS module receives the decoded data and performs video processing, the processed video is transmitted to the virtual VDP module, and the virtual VDP module is used for creating a private video buffer pool, performing buffer rotation with the VPSS module and controlling the frame rate of the whole channel; the frame rate control unit is used for controlling the buffer processing rate of the virtual VDP module to control the processing frame rate of the VPSS module and the decoding frame rate of the VDH module.
9. The method of claim 8, wherein the step of performing buffer rotation by the virtual VDP module and the VPSS module comprises:
the VDH module acquires a public VB block from the public video cache pool to perform decoding processing, and transmits a buffer to the VPSS module after the public VB block is decoded; the VPSS module is used for occupying the public VB block when receiving the buffer, acquiring a private VB block through the virtual VDP module, occupying the private VB block, and transmitting the buffer to the virtual VDP module after the private VB block is processed;
the virtual VDP module is used for occupying the private VB blocks when receiving the buffer, calling a user sub-module user_sub and releasing the public VB blocks transmitted by the VDH module.
10. The method according to claim 8 or 9, wherein the frame rate control unit is disposed in a virtual VDP module and connected to the VPSS module, and the frame rate control unit controls the buffer processing rate of the virtual VDP module by configuring the speed at which the virtual VDP module obtains the private VB block from the private video cache pool;
according to the buffer processing rate of the virtual VDP module, the VPSS module adjusts the processing frame rate to match with the buffer processing rate of the virtual VDP module; the VDH module adjusts the decoding frame rate to match the processing frame rate of the VPSS module according to the processing frame rate of the VPSS module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111508127.0A CN114189712B (en) | 2021-12-10 | 2021-12-10 | Virtual VDP device supporting decoding frame rate control in video monitoring and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111508127.0A CN114189712B (en) | 2021-12-10 | 2021-12-10 | Virtual VDP device supporting decoding frame rate control in video monitoring and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114189712A CN114189712A (en) | 2022-03-15 |
| CN114189712B true CN114189712B (en) | 2024-01-16 |
Family
ID=80604406
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111508127.0A Active CN114189712B (en) | 2021-12-10 | 2021-12-10 | Virtual VDP device supporting decoding frame rate control in video monitoring and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114189712B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007051156A2 (en) * | 2005-10-27 | 2007-05-03 | Qualcomm Incorporated | Video source rate control for video telephony |
| CN101341756A (en) * | 2005-10-27 | 2009-01-07 | 高通股份有限公司 | Video source rate control for video telephony |
| CN105872432A (en) * | 2016-04-21 | 2016-08-17 | 天津大学 | Rapid self-adaptive frame rate conversion device and method |
| CN111787330A (en) * | 2020-06-16 | 2020-10-16 | 眸芯科技（上海）有限公司 | Coding method supporting decoding compression frame buffer self-adaptive distribution and application |
| EP3767915A1 (en) * | 2019-07-17 | 2021-01-20 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Buffer processing method, apparatus, device and computer storage medium |
| CN113115111A (en) * | 2021-03-16 | 2021-07-13 | 广州朗国电子科技有限公司 | Video frame rate control method, system and storage medium for screen recording live broadcast |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100504471B1 (en) * | 2002-08-08 | 2005-08-03 | 엘지전자 주식회사 | Video decoding system |
| US8295344B2 (en) * | 2007-10-22 | 2012-10-23 | Broadcom Corporation | Adaptive rate control for encoding of variable frame rate video sources |
-
2021
- 2021-12-10 CN CN202111508127.0A patent/CN114189712B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007051156A2 (en) * | 2005-10-27 | 2007-05-03 | Qualcomm Incorporated | Video source rate control for video telephony |
| CN101341756A (en) * | 2005-10-27 | 2009-01-07 | 高通股份有限公司 | Video source rate control for video telephony |
| CN105872432A (en) * | 2016-04-21 | 2016-08-17 | 天津大学 | Rapid self-adaptive frame rate conversion device and method |
| EP3767915A1 (en) * | 2019-07-17 | 2021-01-20 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Buffer processing method, apparatus, device and computer storage medium |
| CN111787330A (en) * | 2020-06-16 | 2020-10-16 | 眸芯科技（上海）有限公司 | Coding method supporting decoding compression frame buffer self-adaptive distribution and application |
| CN113115111A (en) * | 2021-03-16 | 2021-07-13 | 广州朗国电子科技有限公司 | Video frame rate control method, system and storage medium for screen recording live broadcast |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114189712A (en) | 2022-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10721282B2 (en) | Media acceleration for virtual computing services | |
| TWI489875B (en) | A video coder | |
| KR101523133B1 (en) | Streaming techniques for video display systems | |
| US9148694B2 (en) | Method and apparatus enabling fast channel change for DSL system | |
| US9307195B2 (en) | Controlling resolution of encoded video | |
| CN101047853B (en) | Server apparatus and video transmission method | |
| CA2908850C (en) | Individual buffer management in video coding | |
| US20040184523A1 (en) | Method and system for providing reduced bandwidth for picture in picture video transmissions | |
| US20100008419A1 (en) | Hierarchical Bi-Directional P Frames | |
| KR101615967B1 (en) | Hypothetical reference decoder for multiview video coding | |
| US8253775B2 (en) | Multipoint conference system, multipoint conference method, and program | |
| US20110002376A1 (en) | Latency Minimization Via Pipelining of Processing Blocks | |
| EP1821544A2 (en) | Encoding apparatus, decoding apparatus, encoding method, decoding method, program of encoding method, program of decoding method, recording medium in which program of encoding method is recorded, and recording medium in which program of decoding method is recorded | |
| JP2007505580A (en) | Method and apparatus for generating graphical and media displays in a thin client | |
| CN102158693A (en) | Method and video receiving system for adaptively decoding embedded video bitstream | |
| CN113542660A (en) | Method, system and storage medium for realizing conference multi-picture high-definition display | |
| KR100703529B1 (en) | Apparatus and method for receiving images of multiple channel in receiving digital multimedia broadcasting | |
| US20110299605A1 (en) | Method and apparatus for video resolution adaptation | |
| KR20220023341A (en) | Sub-pictures and sub-picture sets with level derivation | |
| CN114189712B (en) | Virtual VDP device supporting decoding frame rate control in video monitoring and application | |
| WO2023011408A1 (en) | Multi-window video communication method, device and system | |
| CN113316022B (en) | Video playing method, device, equipment, system and storage medium | |
| KR102259540B1 (en) | 360 degree video streaming based on eye gaze tracking | |
| CN101198050B (en) | Video data processing method and device | |
| CN106658070A (en) | Method and device for redirecting video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |