CN110602523A - VR panoramic live multimedia processing and synthesizing system and method - Google Patents
VR panoramic live multimedia processing and synthesizing system and method Download PDFInfo
- Publication number
- CN110602523A CN110602523A CN201910965954.9A CN201910965954A CN110602523A CN 110602523 A CN110602523 A CN 110602523A CN 201910965954 A CN201910965954 A CN 201910965954A CN 110602523 A CN110602523 A CN 110602523A
- Authority
- CN
- China
- Prior art keywords
- video data
- panoramic
- processed
- editing
- live
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002194 synthesizing effect Effects 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 title claims description 24
- 230000005540 biological transmission Effects 0.000 claims description 9
- 230000006835 compression Effects 0.000 claims description 7
- 238000007906 compression Methods 0.000 claims description 7
- 238000004806 packaging method and process Methods 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 2
- 238000003786 synthesis reaction Methods 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8146—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
- H04N21/8153—Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a VR panoramic live multimedia processing and synthesizing system, which comprises an information acquisition system for acquiring panoramic video data, a stream pushing device for transmitting the panoramic video data and a VR device for playing, and also comprises: the cloud editing system is used for carrying out re-editing processing on the panoramic video data transmitted by the stream pushing device; the stream pulling device is used for transmitting the video data processed by the server; during operation, the information acquisition system firstly acquires panoramic videos, the acquired panoramic video data are transmitted to the cloud editing system through the stream pushing device, the cloud editing system carries out secondary editing processing on the panoramic video data to obtain processed videos, and the processed videos are transmitted to the VR device through the stream pulling device to be played. When the video is uploaded to the server, the video image is processed for the second time, so that the multi-element materials are added to the video, the interestingness and the relativity of the video can be improved, and the user experience and feeling can be improved.
Description
Technical Field
The invention relates to the technical field of network live broadcast, in particular to a VR panoramic live broadcast multimedia processing and synthesizing system and method.
Background
Virtual Reality (abbreviated as VR) is a new practical technology developed in the 20 th century. The virtual reality technology comprises a computer, electronic information and simulation technology, and the basic realization mode is that the computer simulates a virtual environment so as to provide people with environmental immersion. With the continuous development of social productivity and scientific technology, VR technology is increasingly in great demand in various industries. The VR technology has made great progress and gradually becomes a new scientific and technical field.
Audience generally can only follow the angle that the camera was shot and watch live in the ordinary live video, can not all-round or according to oneself hobby understanding the anchor ambient condition, and virtual reality technique can satisfy this demand for audience can watch live from each angle as desired, has strengthened user experience, and the sense of participation also strengthens greatly, consequently combines together virtual reality and live video, is the development trend of live in the future.
The VR live broadcast in the present stage mainly refers to 360-degree panoramic live broadcast, that is, the realization of VR live broadcast video requires the acquisition of panoramic materials, the acquisition of VR panoramic video generally depends on a plurality of wide-angle lenses, and then the VR live broadcast video is spliced into a complete panoramic video through a computer graphics algorithm.
The existing VR live broadcast cannot realize an editing function in the playing process, cannot add various elements in the experience process, and improves the functionality and entertainment of video live broadcast.
Disclosure of Invention
In order to solve the above problems, an object of the present invention is to provide a VR panoramic live multimedia processing and synthesizing system, which can reprocess a video, so that the video obtained by a VR end can load multiple data elements, and solve the problems of monotonous picture and low client stickiness due to the fact that the video received by the existing VR end cannot be edited. A VR panoramic live multimedia processing and synthesizing method is also provided.
The invention is realized by the following technical scheme:
the utility model provides a VR panorama live multimedia processing synthesis system, includes the information acquisition system that is used for gathering panoramic video data, is used for transmitting panoramic video data's plug flow device and the VR device that is used for the broadcast, still includes:
the cloud editing system is used for carrying out re-editing processing on the panoramic video data transmitted by the stream pushing device;
the stream pulling device is used for transmitting the video data processed by the server; during operation, the information acquisition system firstly acquires panoramic videos, the acquired panoramic video data are transmitted to the cloud editing system through the stream pushing device, the cloud editing system carries out secondary editing processing on the panoramic video data to obtain processed videos, and the processed videos are transmitted to the VR device through the stream pulling device to be played.
The live video output by the existing VR live system cannot load various multi-component materials, is single in playing style and monotonous in playing picture sense during live broadcasting, and cannot effectively stimulate the watching desire of audiences and arouse the experience enthusiasm of the audiences. When the video image is uploaded to the server, the video image is processed for the second time, so that multiple materials are added to the video, the interestingness and the relativity of the video can be improved, and the user experience and feeling can be improved.
The cloud editing system comprises a storage server and an editing server, wherein the storage server is used for storing materials required in the re-editing processing process, and the editing server is used for editing and synthesizing the materials stored in the storage server on the panoramic video data.
The materials required in the re-editing process include: music material, caption material, audio frequency adjusting material and character and picture material. The materials provided by the invention are not limited to the materials, and the materials can be flexibly selected in multiple aspects according to the actual requirements.
Among the materials stored in the cloud editing system of the present invention, music materials can be used for adding background sound and dubbing to a video, text pictures can be added to a video for vividly interpreting the feeling of human mind, and pictures in the text picture materials can also be used for blurring a designated position of the video. Specifically, during processing, a video source region, a subtitle source region, an audio source region and a character and picture source region exist in a storage server, a stream pushing device transmits video data to the storage server, other materials are uploaded to the storage server through the prior art, and an editing server is used for processing element materials on a video.
Further, the specific process of transmitting the panoramic video data by the stream pushing device is to firstly compress the panoramic video data, then package the panoramic video data, and then transmit the panoramic video data to the cloud editing system by using a real-time message transmission protocol.
Further, the specific process of transmitting the processing video by the stream pulling device is as follows: and packaging the processed video data, and transmitting the processed video data to the VR device by adopting a real-time message transmission protocol.
Before the streaming device performs packet processing on the processed video data, the processing video data needs to be compressed.
The invention aims to solve the problem of high bandwidth pressure in the transmission process by adopting the compression treatment before the flow pushing device and the flow pulling device. Under the condition of playing videos with the same definition in real time, VR live video needs larger network transmission bandwidth than traditional live video, and brings great pressure to the current network bandwidth. In order to solve the problem of large bandwidth pressure, the method for reducing the bandwidth pressure is different from the prior mode of reducing the bandwidth pressure by VR live broadcasting, the method for reducing the bandwidth pressure fully depends on the logical relationship among a stream pushing device, a cloud editing system, a stream pulling device and a VR device, video data is compressed before the stream pushing device packages, so that the bandwidth occupation uploaded to the cloud editing system is reduced, the uploading speed is improved, and the same is true during stream pulling.
In addition, the information acquisition system can comprise a panoramic camera device.
The VR panoramic live multimedia processing and synthesizing method comprises the steps of collecting video data in multiple angles, transmitting the video data to a cloud editing system through plug-flow transmission, editing the video data into processed video data through cloud editing, transmitting the processed video data to a VR terminal through pull-flow transmission, and playing the processed video data through the VR terminal.
Furthermore, when video data is streamed, compression is required before packetization.
Furthermore, when the processed video data is transmitted via pull stream, compression is required before being packaged.
Compared with the prior art, the invention has the following advantages and beneficial effects:
when the video is uploaded to the server, the video image is processed for the second time, so that the multi-element materials are added to the video, the interestingness and the relativity of the video can be improved, and the user experience and feeling can be improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
FIG. 1 is a schematic view of the structure of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not meant to limit the present invention.
Example 1
As shown in fig. 1, a VR panorama live multimedia processing and synthesizing system includes an information acquisition system for acquiring panorama video data, a stream pushing device for transmitting panorama video data, and a VR device for playing, and further includes:
the cloud editing system is used for carrying out re-editing processing on the panoramic video data transmitted by the stream pushing device;
the stream pulling device is used for transmitting the video data processed by the server; during operation, the information acquisition system firstly acquires panoramic videos, the acquired panoramic video data are transmitted to the cloud editing system through the stream pushing device, the cloud editing system carries out secondary editing processing on the panoramic video data to obtain processed videos, and the processed videos are transmitted to the VR device through the stream pulling device to be played.
The information acquisition system may be a panoramic camera when implemented.
The cloud editing system comprises a storage server and an editing server, wherein the storage server is used for storing materials required in the re-editing processing process, and the editing server is used for editing and synthesizing the materials stored in the storage server on the panoramic video data.
The materials required in the re-editing process include: music material, caption material, audio frequency adjusting material and character and picture material. The materials provided by the invention are not limited to the materials, and the materials can be flexibly selected in multiple aspects according to the actual requirements.
In implementation, among the materials stored in the cloud editing system, music materials can be used for adding background sound and dubbing to videos, text pictures can be added to the videos for vividly explaining the feeling of human mind, and the pictures in the text picture materials can also be used for blurring designated positions of the videos. Specifically, during processing, a video source region, a subtitle source region, an audio source region and a character and picture source region exist in a storage server, a stream pushing device transmits video data to the storage server, other materials are uploaded to the storage server through the prior art, and an editing server is used for processing element materials on a video.
Further, the specific process of transmitting the panoramic video data by the stream pushing device is to firstly compress the panoramic video data, then package the panoramic video data, and then transmit the panoramic video data to the cloud editing system by using a real-time message transmission protocol.
Further, the specific process of transmitting the processing video by the stream pulling device is as follows: and packaging the processed video data, and transmitting the processed video data to the VR device by adopting a real-time message transmission protocol.
Before the streaming device performs packet processing on the processed video data, the processing video data needs to be compressed. Can be carried out by adopting the existing compression processing technology.
Example 2
The VR panoramic live broadcast multimedia processing and synthesizing method comprises the steps of collecting video data in multiple angles, transmitting the video data to a cloud editing system through plug flow, editing the video data into processed video data through cloud editing, transmitting the processed video data to a VR terminal through pull flow, and playing the processed video data through the VR terminal.
Furthermore, when video data is streamed, compression is required before packetization.
Furthermore, when the processed video data is transmitted via pull stream, compression is required before being packaged.
In the present invention, the prior art is not described in detail.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (9)
1. The utility model provides a VR panorama live multimedia processing synthesis system, is including the information acquisition system that is used for gathering panoramic video data, the VR device that is used for transmitting panoramic video data's plug flow device and is used for the broadcast which characterized in that still includes:
the cloud editing system is used for carrying out re-editing processing on the panoramic video data transmitted by the stream pushing device;
the stream pulling device is used for transmitting the video data processed by the server; during operation, the information acquisition system firstly acquires panoramic videos, the acquired panoramic video data are transmitted to the cloud editing system through the stream pushing device, the cloud editing system carries out secondary editing processing on the panoramic video data to obtain processed videos, and the processed videos are transmitted to the VR device through the stream pulling device to be played.
2. The VR live streaming multimedia processing and synthesizing system of claim 1, wherein the cloud editing system includes a storage server and an editing server, the storage server is configured to store materials required in the re-editing process, and the editing server is configured to edit and synthesize the materials stored in the storage server onto the panoramic video data.
3. The VR panorama live multimedia processing and synthesizing system of claim 2, wherein re-editing material needed in the processing comprises: music material, caption material, audio frequency adjusting material and character and picture material.
4. The VR live panoramic multimedia processing and synthesizing system of claim 1, wherein the streaming device transmits the panoramic video data by compressing the panoramic video data, packetizing the panoramic video data, and transmitting the panoramic video data to the cloud editing system using a real-time messaging protocol.
5. The VR panoramic live multimedia processing and synthesizing system of claim 1, wherein the streaming device transmits the processed video in a specific process: and packaging the processed video data, and transmitting the processed video data to the VR device by adopting a real-time message transmission protocol.
6. The VR live panoramic multimedia processing and synthesizing system of claim 5 wherein the streaming device compresses the processed video data before packetizing the processed video data.
7. A VR panoramic live multimedia processing and synthesizing method is characterized in that after video data are collected in multiple angles, the video data are firstly transmitted to a cloud editing system through plug flow, the video data are edited and processed into processed video data through cloud editing, the processed video data are transmitted to a VR terminal through pull flow, and the processed video data are played through the VR terminal.
8. The VR panorama live multimedia processing and synthesizing method of claim 7, wherein video data needs to be compressed before being packetized when being streamed.
9. The VR live panoramic multimedia processing and synthesizing method of claim 7, wherein compression is required before packetization when streaming processed video data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910965954.9A CN110602523A (en) | 2019-10-12 | 2019-10-12 | VR panoramic live multimedia processing and synthesizing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910965954.9A CN110602523A (en) | 2019-10-12 | 2019-10-12 | VR panoramic live multimedia processing and synthesizing system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110602523A true CN110602523A (en) | 2019-12-20 |
Family
ID=68866717
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910965954.9A Pending CN110602523A (en) | 2019-10-12 | 2019-10-12 | VR panoramic live multimedia processing and synthesizing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110602523A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111083395A (en) * | 2019-12-27 | 2020-04-28 | 卓望数码技术（深圳）有限公司 | Panoramic video processing method and system |
| CN111355966A (en) * | 2020-03-05 | 2020-06-30 | 上海乐杉信息技术有限公司 | Surrounding free visual angle live broadcast method and system |
| TWI838727B (en) | 2022-03-30 | 2024-04-11 | 大陸商奧圖碼數碼科技(上海)有限公司 | Multi-resource editing system and multi-resource editing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932608A (en) * | 2012-11-16 | 2013-02-13 | 成都索贝数码科技股份有限公司 | Digital video processing and cataloguing system and method based on cloud edition technology |
| CN107205122A (en) * | 2017-08-03 | 2017-09-26 | 哈尔滨市舍科技有限公司 | The live camera system of multiresolution panoramic video and method |
| CN108447129A (en) * | 2017-12-27 | 2018-08-24 | 广东鸿威国际会展集团有限公司 | A kind of online three-dimensional editing method and system |
| CN108769717A (en) * | 2018-08-22 | 2018-11-06 | 西安凯联电子科技有限公司 | A kind of new media live broadcast system and live broadcasting method |
| CN108833892A (en) * | 2018-05-28 | 2018-11-16 | 徐州昇科源信息技术有限公司 | A kind of VR live broadcast system |
-
2019
- 2019-10-12 CN CN201910965954.9A patent/CN110602523A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102932608A (en) * | 2012-11-16 | 2013-02-13 | 成都索贝数码科技股份有限公司 | Digital video processing and cataloguing system and method based on cloud edition technology |
| CN107205122A (en) * | 2017-08-03 | 2017-09-26 | 哈尔滨市舍科技有限公司 | The live camera system of multiresolution panoramic video and method |
| CN108447129A (en) * | 2017-12-27 | 2018-08-24 | 广东鸿威国际会展集团有限公司 | A kind of online three-dimensional editing method and system |
| CN108833892A (en) * | 2018-05-28 | 2018-11-16 | 徐州昇科源信息技术有限公司 | A kind of VR live broadcast system |
| CN108769717A (en) * | 2018-08-22 | 2018-11-06 | 西安凯联电子科技有限公司 | A kind of new media live broadcast system and live broadcasting method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111083395A (en) * | 2019-12-27 | 2020-04-28 | 卓望数码技术（深圳）有限公司 | Panoramic video processing method and system |
| CN111355966A (en) * | 2020-03-05 | 2020-06-30 | 上海乐杉信息技术有限公司 | Surrounding free visual angle live broadcast method and system |
| TWI838727B (en) | 2022-03-30 | 2024-04-11 | 大陸商奧圖碼數碼科技(上海)有限公司 | Multi-resource editing system and multi-resource editing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106789991B (en) | Multi-person interactive network live broadcast method and system based on virtual scene | |
| CN107846633B (en) | Live broadcast method and system | |
| CN111818004B (en) | Cloud game live broadcast method and system and computer readable storage medium | |
| CN106992959B (en) | 3D panoramic audio and video live broadcast system and audio and video acquisition method | |
| WO2016150317A1 (en) | Method, apparatus and system for synthesizing live video | |
| CN105227963B (en) | Streaming Media collection automatic identification direction and the method and system adjusted are carried out to terminal | |
| US20020154691A1 (en) | System and process for compression, multiplexing, and real-time low-latency playback of networked audio/video bit streams | |
| CN105306468A (en) | Method for real-time sharing of synthetic video data and anchor client side | |
| KR100889367B1 (en) | System and Method for Realizing Vertual Studio via Network | |
| CN107241654A (en) | A kind of high in the clouds accelerates render farm panorama game live broadcast system and method | |
| CN112235585B (en) | Live broadcasting method, device and system for virtual scene | |
| TW200418328A (en) | Instant video conferencing method, system and storage medium implemented in web game using A/V synchronization technology | |
| CN106331880A (en) | Information processing method and information processing system | |
| EP3792731A1 (en) | Multimedia information transmission method and apparatus, and terminal | |
| TW589891B (en) | Real-time video conferencing method, system and storage medium in web game | |
| CN113891117B (en) | Immersion medium data processing method, device, equipment and readable storage medium | |
| CN1534503A (en) | Method of realizing real time image sound talks in network game, system and storage medium thereof | |
| CN110602523A (en) | VR panoramic live multimedia processing and synthesizing system and method | |
| CN107317815A (en) | A kind of method and device, storage medium and the terminal of video superposition | |
| CN109040818B (en) | Audio and video synchronization method, storage medium, electronic equipment and system during live broadcasting | |
| CN114286021B (en) | Rendering method, rendering device, server, storage medium, and program product | |
| CN106507115B (en) | Coding/decoding method, device and the terminal device of VR video based on iOS device | |
| CN112153401B (en) | Video processing method, communication device and readable storage medium | |
| CN109862385B (en) | Live broadcast method and device, computer readable storage medium and terminal equipment | |
| CN109756683A (en) | Panorama audio-video method for recording, device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191220 |
|
| RJ01 | Rejection of invention patent application after publication |