CN114513675A - Construction method of panoramic video live broadcast system - Google Patents

Construction method of panoramic video live broadcast system Download PDF

Info

Publication number
CN114513675A
CN114513675A CN202210001527.0A CN202210001527A CN114513675A CN 114513675 A CN114513675 A CN 114513675A CN 202210001527 A CN202210001527 A CN 202210001527A CN 114513675 A CN114513675 A CN 114513675A
Authority
CN
China
Prior art keywords
video
panoramic
images
live broadcast
nvr
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210001527.0A
Other languages
Chinese (zh)
Inventor
吴军
易博
罗瀛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin University of Electronic Technology
Original Assignee
Guilin University of Electronic Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin University of Electronic Technology filed Critical Guilin University of Electronic Technology
Priority to CN202210001527.0A priority Critical patent/CN114513675A/en
Publication of CN114513675A publication Critical patent/CN114513675A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/78—Television signal recording using magnetic recording
    • H04N5/781—Television signal recording using magnetic recording on disks or drums

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention relates to the technical field of live video, in particular to a construction method of a panoramic live video system, through the establishment of the local area network, the multiple cameras, the NVR and the video processing equipment are connected to form a wireless communication system, so that the video processing equipment can acquire real-time or playback streams of the multiple cameras from the NVR, decoding the obtained video stream to obtain a plurality of images, generating corresponding panoramic images through video splicing, and the panoramic image is arranged on a Mat packaged curtain to realize the rearrangement of multiple paths of images, and finally, based on an FFmpeg coding tool modified by the H.265 standard, the method comprises the steps of carrying out hardware coding on an obtained panoramic image, setting video related parameters, pushing a video stream by using a modified streaming media server after the video coding, realizing multi-channel video input and single-channel video output, and solving the technical problems of complex composition and poor live broadcast effect of the existing panoramic live broadcast system.

Description

Construction method of panoramic video live broadcast system
Technical Field
The invention relates to the technical field of live video, in particular to a construction method of a panoramic live video system.
Background
In recent years, with the continuous development and improvement of networks and video processing technologies, the video live broadcast technology starts to enter the field of view of the public, promotes the rise of the live broadcast industry, and is concerned and used by more and more practitioners. The traditional video live broadcast mode is usually shot by a camera, has the defects of low resolution and small visual angle, and solves the problem in the past by moving the camera, but the blind area is easy to appear because a small-range image is obtained at a certain moment. In the panoramic live broadcast, a plurality of cameras are arranged on one plane, captured video images are spliced into a panoramic video by using an algorithm, and then the panoramic video is transmitted in real time through a streaming media protocol.
The high-resolution panoramic live broadcast has the difficulties of real-time performance, the large time overhead is needed for panoramic video coding and decoding, splicing calculation and video pushing, how to shorten the video processing time, and the realization of real-time video splicing and transmission is the current application difficulty, wherein one of the important research directions is to use the video H.265 coding standard so as to reduce the video capacity and improve the fluency during video pushing under the condition of unchanged image quality; in addition, the traditional live video broadcasting has no storage function, can only be watched in real time, cannot be played back, and can realize video storage by using an NVR (network video recorder), but the live video broadcasting effect is influenced by wired connection.
Disclosure of Invention
The invention aims to provide a construction method of a panoramic video live broadcast system, and aims to solve the technical problems that the existing panoramic live broadcast system is complex in construction and poor in live broadcast effect.
In order to achieve the above object, the present invention provides a method for constructing a panoramic video live broadcast system, comprising the following steps:
erecting a wireless local area network, and accessing the NVR, the multi-path panoramic camera and the computer to the wireless local area network;
acquiring a plurality of paths of videos from the NVR and decoding to obtain a plurality of YUV format images;
the YUV format images are spliced after being converted into formats, and a single panoramic image is generated;
coding and compressing the spliced images based on the H.265 coding standard, and obtaining a path of panoramic video stream in an FLV format after packaging;
and building a streaming media server, and pushing the panoramic video stream to the streaming media server.
The wireless local area network is erected based on a wireless router, and network parameters of the NVR, the multi-path panoramic camera and the computer are all set in the same network segment to form a data-communicated wireless communication network.
The method for obtaining the multiple channels of videos from the NVR and decoding the multiple channels of videos to obtain the multiple YUV format images comprises the following steps:
logging in the NVR, selecting a real-time video stream or a playback video stream according to requirements, and if the playback video stream is the playback video stream, setting the starting time and the ending time of the playback video;
reading input multi-channel network video streams, acquiring data information of each channel of video, and realizing the functions of searching, opening and reading video frames of a decoder;
after frame information corresponding to a video is obtained, decoding a buffer of the size of the video frame by applying a decoding frame buffer, and setting a pixel space format, a frame size and a resolution parameter of a buffer block;
and determining a corresponding decoder according to the video frame information, initializing the decoder, setting an initial value, applying for a memory, then opening the decoder, reading video data, starting cyclic decoding, and obtaining YUV image data after decoding.
Wherein the decoder decodes each captured video stream based on FFmpeg.
The method comprises the steps of converting decoded YUV image data into a BGR format, creating a single-path output video curtain, reading lookup table data, performing parallel operation at a GPU (graphics processing unit) end to obtain spliced panoramic images, and rearranging multiple paths of images on the curtain by splicing to generate a single panoramic image.
The lookup table is composed of a header and a plurality of metadata, the number of the metadata is the same as the number of pixel points in the panoramic image, the metadata is composed of pixel point IDs, the number of source images corresponding to the pixel points and four one-dimensional arrays, and the four one-dimensional arrays respectively store the number, the horizontal coordinate, the vertical coordinate and the weight value of a plurality of source images corresponding to the current pixel point.
The streaming media server is built on an Nginx open source tool, and is compatible with the H.265 coding standard by modifying an RTMP module.
And coding and compressing the spliced panoramic image based on the H.265 coding standard, modifying a coding tool and packaging the panoramic video by using an FLV container.
The invention provides a construction method of a panoramic video live broadcast system, which connects a plurality of cameras, an NVR and a video processing device through the construction of a local area network to form a wireless communication system, so that the video processing device can acquire real-time or playback streams of the plurality of cameras from the NVR, decoding the obtained video stream to obtain a plurality of images, generating corresponding panoramic images through video splicing, and the panoramic image is arranged on a Mat packaged curtain to realize the rearrangement of multiple paths of images, and finally, based on an FFmpeg coding tool modified by the H.265 standard, the method comprises the steps of carrying out hardware coding on an obtained panoramic image, setting video related parameters, pushing a video stream by using a modified streaming media server after the video coding, realizing multi-channel video input and single-channel video output, and solving the technical problems of complex composition and poor live broadcast effect of the existing panoramic live broadcast system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a method for constructing a panoramic video live broadcast system according to the present invention.
Fig. 2 is a schematic diagram of the panoramic video live broadcasting system of the present invention.
Fig. 3 is a flow chart of the multi-channel video capture of the present invention.
FIG. 4 is a diagram of a lookup table structure according to the present invention.
Fig. 5 is a video stitching flow diagram of the present invention.
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
Referring to fig. 1, the present invention provides a method for constructing a panoramic video live broadcast system, which includes the following steps:
s1: erecting a wireless local area network, and accessing the NVR, the multi-path panoramic camera and the computer to the wireless local area network;
s2: acquiring a plurality of paths of videos from the NVR and decoding to obtain a plurality of YUV format images;
s3: the YUV format images are spliced after being converted into formats, and a single panoramic image is generated;
s4: coding and compressing the spliced images based on the H.265 coding standard, and obtaining a path of panoramic video stream in an FLV format after packaging;
s5: and building a streaming media server, and pushing the panoramic video stream to the streaming media server.
The wireless local area network is erected based on a wireless router, and network parameters of the NVR, the multi-path panoramic camera and the computer are all set in the same network segment to form a wireless communication network with data communication.
Further, in addition to using a computer as a video processing device, an embedded device including a CPU and a GPU may also be used to perform video processing, and fig. 2 is schematically shown in the configuration of the panoramic video live broadcast system.
The decoder decodes each acquired video stream separately based on FFmpeg.
The method comprises the steps of converting decoded YUV image data into a BGR format, creating a single-path output video curtain, reading lookup table data, performing parallel operation at a GPU (graphics processing unit) end to obtain spliced panoramic images, and rearranging multiple paths of images on the curtain by splicing to generate a single panoramic image.
The lookup table is composed of a header and a plurality of metadata, the number of the metadata is the same as the number of pixel points in the panoramic image, the metadata is composed of pixel point IDs, the number of source images corresponding to the pixel points and four one-dimensional arrays, and the four one-dimensional arrays respectively store the number, the horizontal coordinate, the vertical coordinate and the weight value of a plurality of source images corresponding to the current pixel point.
The streaming media server is built on an Nginx open source tool, and is compatible with the H.265 coding standard by modifying the RTMP module.
And coding and compressing the spliced panoramic image based on the H.265 coding standard, modifying a coding tool and packaging the panoramic video by using an FLV container.
The following is further explained from the individual steps:
s1: erecting a wireless local area network, and accessing the NVR, the multi-path panoramic camera and the computer to the wireless local area network;
specifically, a local area network is built through the WiFi function of the router, and the network configuration of the NVR and the computer is modified based on the panoramic camera, so that the computer can be connected to the NVR through a wireless network, and a real-time video stream shot by the panoramic camera can be acquired from the NVR, or a video in a certain time period can be selected according to requirements.
S2: acquiring a plurality of paths of videos from the NVR and decoding to obtain a plurality of YUV format images;
specifically, as shown in fig. 3, according to actual requirements, multiple real-time video streams or multiple playback video streams shot by the panoramic camera are collected, and the multiple video streams obtained from the NVR are respectively decoded by using FFmpeg to obtain decoded YUV format images, and the further specific steps are as follows:
s201: logging in the NVR, selecting a real-time video stream or a playback video stream according to requirements, and if the playback video stream is the playback video stream, setting the starting time and the ending time of the playback video;
s202: reading input multi-channel network video streams, acquiring data information of each channel of video, and realizing the functions of searching, opening and reading video frames of a decoder;
s203: after frame information corresponding to a video is obtained, decoding a buffer of the size of the video frame by applying a decoding frame buffer, and setting a pixel space format, a frame size and a resolution parameter of a buffer block;
s204: and determining a corresponding decoder according to the video frame information, initializing the decoder, setting an initial value, applying for a memory, then opening the decoder, reading video data, starting cyclic decoding, and obtaining YUV image data after decoding.
S3: the YUV format images are spliced after being converted into formats, and a single panoramic image is generated;
specifically, the decoded YUV image data is converted into a BGR format to prepare for subsequent splicing processing.
Presetting a Mat packaged black output curtain, and reading the quantity, the serial number, the horizontal and vertical coordinate values and the weight value of each pixel point corresponding to the images to be spliced in a file stream mode. After receiving the image and the lookup table data transmitted by the CPU, the GPU is used for simultaneously operating each pixel point, the pixel points near the suture line are gradually merged to realize transition, the input image is rearranged on an output curtain through video splicing to obtain a panoramic image, and the synthesis of one path of panoramic video from multiple paths of input videos is realized. The method comprises the following specific steps:
s301: creating a black curtain with the same resolution as the preset output resolution by utilizing Mat packaging, and rearranging the input multi-channel video on the curtain;
s302: the lookup table is stored in a DAT format file, data information of each pixel point is read at the CPU end in a file stream mode, and the read data are transmitted to the GPU end;
s303: the GPU chunk size is adjusted according to the boundary range of each image in the mapping table, and the grid size is set through the acquired GPU equipment information, so that the parallel computing operation efficiency of the GPU is improved;
s304: utilizing a GPU to simultaneously perform bilinear interpolation operation on all pixel points, and performing gradual-in and gradual-out fusion on the pixel points with nonzero weight values in the lookup table to realize the transition of the suture line;
s305: after the parallel operation is finished, the panoramic image is placed on a pre-created curtain to finish the re-layout of multiple paths of images, and the output of a single panoramic image is realized.
The structure of the read lookup table is shown in fig. 4, the lookup table is composed of a table header and n pieces of metadata, and n is the number of pixels in the panoramic image. The header of the table has 5 parameters, wherein Cam _ num represents the number of the multi-path cameras, dst _ weight and dst _ height are respectively the width and height of the spliced panoramic image, and src _ weight and src _ height are respectively the width and height of the source image. The pixel points of the panoramic image are spliced by a plurality of source images, the source images corresponding to any pixel point and the number of the source images are different, and source image information corresponding to the pixel points in the panoramic image is stored by utilizing metadata. Metadata Ele [ i ]]Numbered by pixel point EleIDiNumber of source images numVid corresponding to pixel pointsiAnd four one-dimensional arrays, four arrays VidIDi[]、PixXi[]、PixYi[]、Weighti[]And respectively storing the serial number id, the abscissa, the ordinate and the weight value of the source image.
S4: coding and compressing the spliced images based on the H.265 coding standard, and obtaining a path of panoramic video stream in an FLV format after packaging;
specifically, as shown in fig. 5, the panoramic image is transmitted to a hardware encoder in the GPU, a hardware encoder environment is initialized, the FFmpeg is modified to support encoding of the h.265 standard video encapsulated by the FLV, the panoramic image is encoded based on the H265 encoding standard, and the FLV is encapsulated to obtain a path of panoramic video. The method comprises the following specific steps:
s401: integrating the FFmpeg modification part into a source code through a patch command, and recompiling the FFmpeg source code to obtain an FFmpeg dynamic library supporting H.265;
s402: and initializing a hardware encoder, and dividing the obtained panoramic image into different encoding units. In the encoding process, for the image frames with less change of the previous and next frames, only the changed image frames are selected to be encoded, so that the video compression efficiency is improved;
s403: a random access coding structure is adopted to remove redundant information of a time domain space domain, so that a prediction image block is obtained, and the real-time performance of video plug flow is also ensured;
s404: allocating an output format, initializing encoder configuration, and setting encoder parameters such as output video width, height, code rate, frame rate, encoding structure type and the like;
s405: and (3) designating a GPU hardware encoder as an encoder used by the single-path panoramic image, writing the encoder into a video file header after the encoder is opened, and writing the related information of the designated format into the video.
S406: and after the data reading is judged, two functions of avcodec _ send _ frame and avcodec _ receive _ packet are called to process the video frame and the video packet. And sequentially sending the video frames to an encoder according to the time stamp sequence, sequentially processing the video frames after the encoder receives the video frames, and sequentially outputting the video frames as packets after buffering or encoding the video frames.
S407: and packaging the coded Packet in an FLV format, and resetting the codecID in the FLV package to meet the condition of subsequent plug flow.
S5: and building a streaming media server, and pushing the panoramic video stream to the streaming media server.
Specifically, the secondary development is carried out on the basis of the nginx-rtmp-module open source module, so that the video of the H.265 coding standard can be compatible. And pushing the coded single-path panoramic video to a streaming media server by modifying the server conf configuration file. The method comprises the following specific steps:
s501: modifying the nginx-rtmp-module, increasing the encoding ID of H.265, and extracting the newly added VPS video parameter set in the H.265, thereby reading the information required by the multi-layer and sub-layer video encoding;
s502: and (3) building a streaming media server supporting H.265 by using the modified RTMP module, modifying a conf configuration file of an Nginx open source tool, setting a monitoring port number to 1935, and setting a database Chunksize in RTMP transmission to 4000.
S503: setting a video decoding time stamp and a display time stamp to be consistent with the input multi-channel video, and preventing the video from being jammed, quickly played and the like;
s504: and pushing the single-path panoramic video to a streaming media server, and watching the video by using any playing end.
While the invention has been described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (7)

1. A construction method of a panoramic video live broadcast system is characterized by comprising the following steps:
erecting a wireless local area network, and accessing the NVR, the multi-path panoramic camera and the computer to the wireless local area network;
acquiring a plurality of paths of videos from the NVR and decoding to obtain a plurality of YUV format images;
the YUV format images are spliced after being converted into formats, and a single panoramic image is generated;
coding and compressing the spliced images based on the H.265 coding standard, and obtaining a path of panoramic video stream in an FLV format after packaging;
and building a streaming media server, and pushing the panoramic video stream to the streaming media server.
2. The method of constructing a panoramic video live broadcast system according to claim 1,
the wireless local area network is erected based on a wireless router, and network parameters of the NVR, the multi-path panoramic camera and the computer are all set in the same network segment to form a wireless communication network with data communication.
3. The method of constructing a panoramic video live broadcast system according to claim 1,
obtaining a plurality of paths of videos from the NVR and decoding the videos to obtain a plurality of YUV format images, wherein the method comprises the following steps:
logging in the NVR, selecting a real-time video stream or a playback video stream according to requirements, and if the playback video stream is the playback video stream, setting the starting time and the ending time of the playback video;
reading input multi-channel network video streams, acquiring data information of each channel of video, and realizing the functions of searching, opening and reading video frames of a decoder;
after frame information corresponding to a video is obtained, decoding a buffer of the size of the video frame by applying a decoding frame buffer, and setting a pixel space format, a frame size and a resolution parameter of a buffer block;
and determining a corresponding decoder according to the video frame information, initializing the decoder, setting an initial value, applying for a memory, then opening the decoder, reading video data, starting cyclic decoding, and obtaining YUV image data after decoding.
4. The method of constructing a panoramic video live broadcast system according to claim 3,
the decoder decodes each acquired video stream separately based on FFmpeg.
5. The method for constructing a panoramic video live broadcast system according to claim 1,
the method comprises the steps of converting decoded YUV image data into a BGR format, creating a single-path output video curtain, reading lookup table data, performing parallel operation at a GPU (graphics processing unit) end to obtain spliced panoramic images, and rearranging multiple paths of images on the curtain by splicing to generate a single panoramic image.
6. The method of constructing a panoramic video live broadcast system according to claim 5,
the lookup table is composed of a header and a plurality of metadata, the number of the metadata is the same as the number of pixel points in the panoramic image, the metadata is composed of pixel point IDs, the number of source images corresponding to the pixel points and four one-dimensional arrays, and the four one-dimensional arrays respectively store the number, the horizontal coordinate, the vertical coordinate and the weight value of a plurality of source images corresponding to the current pixel point.
7. The method of constructing a panoramic video live broadcast system according to claim 1,
the streaming media server is built on an Nginx open source tool, and is compatible with the H.265 coding standard by modifying the RTMP module.
CN202210001527.0A 2022-01-04 2022-01-04 Construction method of panoramic video live broadcast system Pending CN114513675A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210001527.0A CN114513675A (en) 2022-01-04 2022-01-04 Construction method of panoramic video live broadcast system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210001527.0A CN114513675A (en) 2022-01-04 2022-01-04 Construction method of panoramic video live broadcast system

Publications (1)

Publication Number Publication Date
CN114513675A true CN114513675A (en) 2022-05-17

Family

ID=81550119

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210001527.0A Pending CN114513675A (en) 2022-01-04 2022-01-04 Construction method of panoramic video live broadcast system

Country Status (1)

Country Link
CN (1) CN114513675A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024125059A1 (en) * 2022-12-13 2024-06-20 Oppo广东移动通信有限公司 Method and apparatus for processing video data, storage medium, and electronic device

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103020990A (en) * 2012-12-06 2013-04-03 华中科技大学 Moving object detecting method based on graphics processing unit (GPU)
US20130302015A1 (en) * 2011-02-15 2013-11-14 S.I.Sv.El Societa Italiana Per Lo Sviluppo Dell'elettronica S.P.A. Method of Acquisition, Storage and Use of Data Relating to a Three-Dimensional Video Stream, and Video Processing Apparatus Thereof
CN103763479A (en) * 2013-12-31 2014-04-30 深圳英飞拓科技股份有限公司 Splicing device for real-time high speed high definition panoramic video and method thereof
CN103927721A (en) * 2014-04-10 2014-07-16 哈尔滨工程大学 Moving object edge enhancement method based on GPU
CN104639905A (en) * 2015-02-03 2015-05-20 广西智询信息科技有限公司 Wireless video monitoring method suitable for live broadcast on mobile terminal of iOS system
CN106454388A (en) * 2016-09-20 2017-02-22 北京百度网讯科技有限公司 Method and device for determining live broadcast setting information
CN107147916A (en) * 2017-06-09 2017-09-08 苏州蜗牛数字科技股份有限公司 A kind of method for transmitting H.265 encoded video data on the level of the transport layer
CN107948544A (en) * 2017-11-28 2018-04-20 长沙全度影像科技有限公司 A kind of multi-channel video splicing system and method based on FPGA
CN108055472A (en) * 2017-12-21 2018-05-18 长沙全度影像科技有限公司 A kind of real time panoramic live broadcast system and method
CN109961404A (en) * 2017-12-25 2019-07-02 沈阳灵景智能科技有限公司 A kind of high clear video image Enhancement Method based on GPU parallel computation
CN109961389A (en) * 2017-12-25 2019-07-02 沈阳灵景智能科技有限公司 A kind of multi-path high-definition video-splicing method based on GPU parallel computation
CN110278410A (en) * 2019-05-20 2019-09-24 上海澳马信息技术服务有限公司 A kind of more mesh panoramic video joining methods and system
CN111010593A (en) * 2019-11-08 2020-04-14 深圳市麦谷科技有限公司 Method and device for packaging H.265 video data based on FLV format
CN111193877A (en) * 2019-08-29 2020-05-22 桂林电子科技大学 ARM-FPGA (advanced RISC machine-field programmable gate array) cooperative wide area video real-time fusion method and embedded equipment
US10666941B1 (en) * 2016-04-06 2020-05-26 Ambarella International Lp Low bitrate encoding of panoramic video to support live streaming over a wireless peer-to-peer connection
CN111416989A (en) * 2020-04-28 2020-07-14 北京金山云网络技术有限公司 Video live broadcast method and system and electronic equipment
CN111862240A (en) * 2020-07-28 2020-10-30 展讯通信（上海）有限公司 Panoramic camera and calibration method thereof, panoramic image splicing method and storage medium
CN112637551A (en) * 2020-11-18 2021-04-09 合肥市卓迩无人机科技服务有限责任公司 Panoramic data management software system for multi-path 4K quasi-real-time spliced videos

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130302015A1 (en) * 2011-02-15 2013-11-14 S.I.Sv.El Societa Italiana Per Lo Sviluppo Dell'elettronica S.P.A. Method of Acquisition, Storage and Use of Data Relating to a Three-Dimensional Video Stream, and Video Processing Apparatus Thereof
CN103020990A (en) * 2012-12-06 2013-04-03 华中科技大学 Moving object detecting method based on graphics processing unit (GPU)
CN103763479A (en) * 2013-12-31 2014-04-30 深圳英飞拓科技股份有限公司 Splicing device for real-time high speed high definition panoramic video and method thereof
CN103927721A (en) * 2014-04-10 2014-07-16 哈尔滨工程大学 Moving object edge enhancement method based on GPU
CN104639905A (en) * 2015-02-03 2015-05-20 广西智询信息科技有限公司 Wireless video monitoring method suitable for live broadcast on mobile terminal of iOS system
US10666941B1 (en) * 2016-04-06 2020-05-26 Ambarella International Lp Low bitrate encoding of panoramic video to support live streaming over a wireless peer-to-peer connection
CN106454388A (en) * 2016-09-20 2017-02-22 北京百度网讯科技有限公司 Method and device for determining live broadcast setting information
CN107147916A (en) * 2017-06-09 2017-09-08 苏州蜗牛数字科技股份有限公司 A kind of method for transmitting H.265 encoded video data on the level of the transport layer
CN107948544A (en) * 2017-11-28 2018-04-20 长沙全度影像科技有限公司 A kind of multi-channel video splicing system and method based on FPGA
CN108055472A (en) * 2017-12-21 2018-05-18 长沙全度影像科技有限公司 A kind of real time panoramic live broadcast system and method
CN109961389A (en) * 2017-12-25 2019-07-02 沈阳灵景智能科技有限公司 A kind of multi-path high-definition video-splicing method based on GPU parallel computation
CN109961404A (en) * 2017-12-25 2019-07-02 沈阳灵景智能科技有限公司 A kind of high clear video image Enhancement Method based on GPU parallel computation
CN110278410A (en) * 2019-05-20 2019-09-24 上海澳马信息技术服务有限公司 A kind of more mesh panoramic video joining methods and system
CN111193877A (en) * 2019-08-29 2020-05-22 桂林电子科技大学 ARM-FPGA (advanced RISC machine-field programmable gate array) cooperative wide area video real-time fusion method and embedded equipment
CN111010593A (en) * 2019-11-08 2020-04-14 深圳市麦谷科技有限公司 Method and device for packaging H.265 video data based on FLV format
CN111416989A (en) * 2020-04-28 2020-07-14 北京金山云网络技术有限公司 Video live broadcast method and system and electronic equipment
CN111862240A (en) * 2020-07-28 2020-10-30 展讯通信（上海）有限公司 Panoramic camera and calibration method thereof, panoramic image splicing method and storage medium
CN112637551A (en) * 2020-11-18 2021-04-09 合肥市卓迩无人机科技服务有限责任公司 Panoramic data management software system for multi-path 4K quasi-real-time spliced videos

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024125059A1 (en) * 2022-12-13 2024-06-20 Oppo广东移动通信有限公司 Method and apparatus for processing video data, storage medium, and electronic device

Similar Documents

Publication Publication Date Title
US6989868B2 (en) Method of converting format of encoded video data and apparatus therefor
CN110460858B (en) Information processing apparatus and method
EP0917371B1 (en) Moving image composing system
RU2511595C2 (en) Image signal decoding apparatus, image signal decoding method, image signal encoding apparatus, image encoding method and programme
CN112689119B (en) Processing method and device for screen combination of recorded videos in cloud conference
JPH08195960A (en) Method and apparatus for effectively addressing dram in video decompression processor
RU2276470C2 (en) Method for compacting and unpacking video data
TW450004B (en) Programmable filter for removing selected user data from an MPEG-2 bit-stream
JP7177034B2 (en) Method, apparatus and stream for formatting immersive video for legacy and immersive rendering devices
CN107683607A (en) Message processing device and information processing method
WO2019128668A1 (en) Method and apparatus for processing video bitstream, network device, and readable storage medium
CN112511896A (en) Video rendering method and device
JP2008524932A (en) Image decoder with context-based parameter buffer
WO2022021519A1 (en) Video decoding method, system and device and computer-readable storage medium
CN114513675A (en) Construction method of panoramic video live broadcast system
US9584755B2 (en) Endoscope with high definition video recorder/player
CN103686077A (en) Double buffering method applied to realtime audio-video data transmission of 3G wireless network
CN111406404B (en) Compression method, decompression method, system and storage medium for obtaining video file
CN105376585B (en) A kind of method that group of picture closes raising transmission of video speed
CN112511768B (en) Multi-picture synthesis method, device, equipment and storage medium
US11889220B2 (en) Method for coding space information in continuous dynamic images
CN105657448A (en) Method, device and system for forwarding encoded video streams
KR100970992B1 (en) System and method for multiplexing stereoscopic high-definition video through gpu acceleration and transporting the video with light-weight compression and storage media having program source thereof
KR101347112B1 (en) Time-lapse image editing apparatus and time-lapse image editing method
CN104469399A (en) Method for macro block SKIP type selection in spatial resolution video transcoding

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination