CN113542896B - Video live broadcast method, equipment and medium of free view angle - Google Patents
Video live broadcast method, equipment and medium of free view angle Download PDFInfo
- Publication number
- CN113542896B CN113542896B CN202110546103.8A CN202110546103A CN113542896B CN 113542896 B CN113542896 B CN 113542896B CN 202110546103 A CN202110546103 A CN 202110546103A CN 113542896 B CN113542896 B CN 113542896B
- Authority
- CN
- China
- Prior art keywords
- mobile phone
- video
- cloud
- live broadcast
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000000007 visual effect Effects 0.000 claims abstract description 9
- 238000012360 testing method Methods 0.000 claims description 26
- 238000012937 correction Methods 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/61—Network physical structure; Signal processing
- H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
- H04N21/6125—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/61—Network physical structure; Signal processing
- H04N21/6156—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network
- H04N21/6175—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network involving transmission via Internet
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides a video live broadcast method of a free view angle, which comprises the steps of receiving audio and video data, synchronizing audio and video stream data, receiving a live broadcast instruction of the free view angle and live broadcasting of the free view angle. According to the video live broadcasting method of the free view angle, shooting of shooting equipment is controlled through the cloud live broadcasting console according to the unified remote control instruction input by the live broadcasting, the operation flow of the shooting equipment in video live broadcasting is simplified, in addition, audio and video data shot by the shooting equipment are uploaded to the cloud live broadcasting console, and the audio and video data are allowed to be uploaded to one cloud mobile phone, so that a shooting manager can conveniently monitor shooting working conditions through low-definition streams forwarded by the cloud mobile phone; the high-definition data is stored in the cloud, data support is provided for preparing for later-stage production of the wonderful moment, the function of free visual angle playing of the video data at the wonderful moment is provided, the experience of a user is improved, the trouble that the live video broadcast is limited by the condition of the live video broadcast field is eliminated, and the live video broadcast efficiency is improved.
Description
Technical Field
The present invention relates to the field of live broadcast, and in particular, to a method, an apparatus, and a medium for live video broadcast with a free viewing angle.
Background
With the vigorous development of the internet, live broadcasting can rapidly and accurately transmit live information, and more people watch live broadcasting through websites or mobile phones due to strong presence, so that live broadcasting is also more and more popular. The live broadcast is essentially video broadcast, the current live broadcast is mainly a simple structure of local video, for a viewing end, pictures seen by a viewer are all live broadcast with a single view angle or live broadcast of different scenes through different windows, but the video seen by the viewer from the single live broadcast window is a live broadcast picture with a single view angle, the live broadcast picture with other view angles needs to be watched, the current live broadcast window needs to be closed, and a new live broadcast window needs to be clicked to finish switching of images with different view angles, and a large number of entity equipment is needed in the current live broadcast process, such as a synchronous cable for realizing data transmission, a live broadcast guiding device and a live broadcast control device for realizing live broadcast control, and the live broadcast control of a plurality of entity devices is limited by site conditions.
In conclusion, the current video live broadcast is limited by live broadcast site conditions, so that the efficiency of the video live broadcast and the experience of viewers are reduced.
Disclosure of Invention
In order to overcome the defects of the prior art, one of the purposes of the invention is to provide a video live broadcast method with a free view angle, which can solve the problems that the current video live broadcast is limited by the condition of a live broadcast field, and the efficiency of the video live broadcast and the experience of a viewer are reduced.
The second objective of the present invention is to provide an electronic device, which can solve the problems that the current live video is limited by the condition of the live video field, and the efficiency of live video and the experience of viewers are reduced.
It is a third object of the present invention to provide a computer readable storage medium, which can solve the problems that the current video live broadcast is limited by the condition of the live broadcast field, and the efficiency of the video live broadcast and the experience of the viewer are reduced.
One of the purposes of the invention is realized by adopting the following technical scheme:
the method is applied to a cloud live broadcast service system, the cloud live broadcast service system comprises a cloud mobile phone server, a cloud live broadcast control console, a push-pull stream server, a cloud broadcast guiding platform and shooting equipment which is arranged at different view angles corresponding to shooting objects and has included angles between adjacent shooting equipment within a preset included angle range, the cloud mobile phone server is provided with cloud mobile phones with the same quantity as the shooting equipment, and each cloud mobile phone is connected with only the shooting equipment, and the method comprises the following steps:
receiving audio and video data, wherein the cloud mobile phone server controls each cloud mobile phone to receive an audio and video data packet of a shooting object corresponding to a visual angle uploaded by corresponding shooting equipment and store the audio and video data packet in a corresponding buffer queue, and each audio and video data packet received by each cloud mobile phone contains first timestamp information taking the shooting equipment time as a reference;
synchronizing the audio and video stream data, and controlling each cloud mobile phone to convert the first time stamp information into second time stamp information taking the time of the cloud mobile phone server as a reference by the cloud mobile phone server according to a multi-channel audio and video synchronization instruction by the cloud mobile phone server;
receiving a free view live broadcast instruction, wherein a cloud mobile phone server receives a live broadcast view instruction sent by a cloud live broadcast control console, and the free view live broadcast instruction is input to the cloud live broadcast control console through a terminal device by a guide broadcast;
and when the free view live broadcast instruction is a highlight free view instruction, the highlight free view instruction is a time point when the highlight is required to be played in the free view, the cloud mobile phone server controls the cloud mobile phone to send all audio and video data corresponding to the highlight to the preset image processing server, the preset image processor carries out image reconstruction on images in the audio and video data corresponding to the highlight according to a preset image reconstruction algorithm, video data containing a virtual viewpoint is obtained and recorded as highlight video data, and the highlight video data is sent to the cloud broadcasting guide platform by the preset image processor and pushed to the live broadcast platform by the cloud broadcasting guide platform according to the corresponding time point for playing.
Further, the free view live broadcast instruction is a whole free view instruction, the whole free view instruction is that video data in audio and video data corresponding to any moment needs free view live broadcast, and the free view live broadcast specifically includes: the cloud mobile phone server controls the cloud mobile phone to send all audio and video data to a preset image processing server, and image reconstruction is carried out on images in video data uploaded by all shooting devices at each moment according to a preset image reconstruction algorithm, so that video data containing virtual viewpoints are obtained, the video data after image reconstruction at each moment is sent to a push-pull streaming server, and the push-pull streaming server pushes video data to a live broadcasting platform for video playing at free viewpoints.
Further, the step of generating a preset clock correction value further includes the steps of:
calculating a first clock difference value, wherein the cloud mobile phone server sends a first test data packet to the cloud mobile phone, receives a first reply data packet sent by the cloud mobile phone according to the first test data packet, calculates a time difference value between the cloud mobile phone and the cloud mobile phone server according to the sending time and the receiving time of the first test data packet and the first reply data packet, and records the time difference value as the first clock difference value;
calculating a second clock difference value, wherein the cloud mobile phone server controls the cloud mobile phone to send a second test data packet to the corresponding shooting equipment and controls the cloud mobile phone to receive a second reply data packet sent by the corresponding shooting equipment according to the second test data packet, calculates the time difference value between the shooting equipment and the corresponding cloud mobile phone according to the sending time and the receiving time of the second test data packet and the second reply data packet, and marks the time difference value as the second clock difference value;
and calculating a preset clock correction value, and calculating the time difference between the shooting equipment and the cloud mobile phone server according to the first clock difference value and the second clock difference value and recording the time difference as the preset clock correction value.
Further, the video data after image reconstruction is sent to a push-pull streaming server, and the push-pull streaming server pushes the video data to a live broadcast platform for video playing at a free view angle specifically comprises the following steps: and sending the video data subjected to image reconstruction to a push-pull streaming server, pushing the video data to a corresponding live broadcast platform by the push-pull streaming server according to live broadcast requests of different viewer addresses to play video at a free view angle, and sending the live broadcast requests after the live broadcast platform establishes connection with the push-pull streaming server according to the pre-stored push-pull streaming addresses.
Further, the live broadcast platform plays video with a free view angle, and a viewer logs in through a mobile phone terminal or a computer or a television to watch the video.
The method is applied to a cloud live broadcast service system, the cloud live broadcast service system comprises a cloud mobile phone server, a cloud live broadcast control console, a push-pull stream server, a cloud broadcast guiding platform and shooting equipment which is arranged at different view angles corresponding to shooting objects and has included angles between adjacent shooting equipment within a preset included angle range, the cloud mobile phone server is provided with cloud mobile phones with the same quantity as the shooting equipment, and each cloud mobile phone is connected with only the shooting equipment, and the method comprises the following steps:
the cloud live broadcast control console controls shooting equipment arranged at the corresponding position of different visual angles of a shooting object to shoot audio and video according to a remote control instruction input by the guide broadcast, and uploads the shot audio and video data to a corresponding cloud mobile phone;
the method comprises the steps that a multipath audio and video synchronization instruction is sent, and a cloud live broadcast control console obtains the multipath audio and video synchronization instruction input by a terminal device in a guide mode and sends the multipath audio and video synchronization instruction to a cloud mobile phone server;
and sending the free view live broadcast instruction, and the cloud live broadcast control console acquires the free view live broadcast instruction input by the terminal equipment and sends the live broadcast view instruction to the cloud mobile phone server.
The method is applied to a cloud live broadcast service system, the cloud live broadcast service system comprises a cloud mobile phone server, a cloud live broadcast control console, a push-pull stream server, a cloud broadcast guiding platform and shooting equipment which is arranged at different view angles corresponding to shooting objects and has included angles between adjacent shooting equipment within a preset included angle range, the cloud mobile phone server is provided with cloud mobile phones with the same quantity as the shooting equipment, and each cloud mobile phone is connected with only the shooting equipment, and the method comprises the following steps:
receiving the video data at the wonderful moment, and the cloud broadcasting guide station receives and stores the video data at the wonderful moment sent by the preset image processor;
and pushing the video data at the wonderful moment, and sending the video data at the wonderful moment to a push-pull streaming server by the cloud broadcasting guide platform according to the time point corresponding to the wonderful moment, and pushing the video data to a live broadcasting platform by the push-pull streaming server for broadcasting.
The second purpose of the invention is realized by adopting the following technical scheme:
an electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising a live video method for performing one of the freeview methods in the present application.
The third purpose of the invention is realized by adopting the following technical scheme:
a computer-readable storage medium having stored thereon a computer program, characterized by: the computer program is executed by a processor to perform a live video method of free view in the present application.
Compared with the prior art, the invention has the beneficial effects that: according to the video live broadcasting method of the free view angle, shooting of shooting equipment is controlled through the cloud live broadcasting console according to the unified remote control instruction input by the live broadcasting, the operation flow of the shooting equipment in video live broadcasting is simplified, in addition, audio and video data shot by the shooting equipment are uploaded to the cloud live broadcasting console, and the audio and video data are allowed to be uploaded to one cloud mobile phone, so that a shooting manager can conveniently monitor shooting working conditions through low-definition streams forwarded by the cloud mobile phone; the cloud mobile phone server receives the cloud live control console installed in the broadcasting-guiding terminal equipment and the broadcasting-guiding platform to control the live process, so that the broadcasting in the whole video live can be remotely controlled in any place, the trouble that the video live is limited by the condition of a live field is eliminated, the efficiency of the video live is improved, and in addition, video data containing virtual viewpoints after image reconstruction is sent to the push-pull streaming server and pushed to the live platform for video playing at the free viewing angles by the push-pull streaming server, so that a viewer can watch live pictures at different viewing angles in a single window, and the experience of the viewer is enhanced.
The foregoing description is only an overview of the present invention, and is intended to provide a better understanding of the present invention, as it is embodied in the following description, with reference to the preferred embodiments of the present invention and the accompanying drawings. Specific embodiments of the present invention are given in detail by the following examples and the accompanying drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
fig. 1 is a flow chart of a video live broadcast method with a free view angle.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and detailed description, wherein it is to be understood that, on the premise of no conflict, the following embodiments or technical features may be arbitrarily combined to form new embodiments.
The video live broadcast method of the free view angle is applied to a framework of a cloud live broadcast control system, wherein the framework comprises a cloud mobile phone server, a cloud live broadcast control console, a push-pull stream server, a cloud live broadcast guide table and shooting equipment arranged at positions corresponding to different view angles of a shooting object, the shooting equipment is placed in a preset included angle range according to the included angle between adjacent shooting equipment, the preset included angle is 6-10 degrees, namely the included angle degree between the adjacent shooting equipment at different view angles is 6-10 degrees. The cloud mobile phone server is provided with cloud mobile phones with the same quantity as the shooting equipment, each cloud mobile phone is connected with the unique shooting equipment, and the cloud live broadcast control console can be arranged in any mobile phone or tablet personal computer or desktop computer, so that a director can control the whole live broadcast even if the director is not in a live broadcast scene, and the cloud director can also be arranged in any mobile phone or tablet personal computer or desktop computer, so that the control of a live broadcast controller on the live broadcast in the traditional live broadcast process is prevented from being limited by a live broadcast field.
As shown in fig. 1, a live video broadcasting method with a free view angle in the present application includes the following specific steps:
and the cloud live broadcast control console controls shooting equipment arranged at the corresponding position of different visual angles of the shooting object to shoot audio and video according to the remote control instruction input by the guide broadcast, and uploads the shot audio and video data to the corresponding cloud mobile phone. In this embodiment, the shooting devices are cameras or mobile phones, and the shooting devices are all provided with operation instructions for executing a live cloud control console and a live cloud control service program connected with the live cloud control console, and the live cloud control console sends remote control instructions to corresponding shooting devices through the live cloud control service program, and the shooting devices shoot shooting objects according to the corresponding remote control instructions. In this embodiment, each photographing device establishes communication connection with a corresponding cloud mobile phone in the cloud mobile phone server, the photographing device uploads the photographed audio and video data to the corresponding cloud mobile phone in real time, and when the photographing device uploads the audio and video data, a timestamp is recorded for each frame of the photographed audio and video data with the time of the photographing device as a reference, so as to obtain first timestamp information corresponding to the audio and video data, and the first timestamp information is added in a header information area of the audio and video data. The head information area of the audio and video data received by each cloud mobile phone contains corresponding first timestamp information.
And receiving the audio and video data, controlling each cloud mobile phone to receive the audio and video data packet of the shooting object corresponding to the visual angle uploaded by the corresponding shooting equipment by the cloud mobile phone server, and storing the audio and video data packet into the corresponding buffer queue, wherein the audio and video data packet received by each cloud mobile phone comprises first timestamp information taking the time of the shooting equipment as a reference.
And sending the multi-channel audio and video synchronization instruction, and the cloud live broadcast control console acquires the multi-channel audio and video synchronization instruction input by the terminal equipment and sends the multi-channel audio and video synchronization instruction to the cloud mobile phone server.
And receiving a plurality of audio and video synchronization instructions, wherein the cloud mobile phone server receives the plurality of audio and video synchronization instructions sent by the cloud live broadcast control console, and the plurality of audio and video synchronization instructions are input to the cloud live broadcast control console by a guide broadcast through terminal equipment (mobile phone or tablet personal computer and other terminal identification).
Generating a preset clock correction value, wherein the generating of the preset clock correction value specifically comprises the following steps:
calculating a first clock difference value, wherein in the embodiment, the cloud mobile phone is respectively connected with a cloud mobile phone server and shooting equipment; the cloud mobile phone server sends a first test data packet to the cloud mobile phone, receives a first reply data packet sent by the cloud mobile phone according to the first test data packet, calculates a time difference between the cloud mobile phone and the cloud mobile phone server according to the sending time and the receiving time of the first test data packet and the first reply data packet, and records the time difference as a first clock difference; the method comprises the following steps: the cloud mobile phone server sends a first test data packet to the cloud mobile phone and receives a first reply data packet sent by the cloud mobile phone according to the first test data packet, the time of the cloud mobile phone server sending the first test data packet is recorded as first sending time, the time of the cloud mobile phone receiving the first test data packet is used as first receiving time, the time of the cloud mobile phone sending the first reply data packet is recorded as second sending time, the time of the cloud mobile phone server receiving the first reply data packet is recorded as second receiving time, and a first clock difference value between the cloud mobile phone and the cloud mobile phone server is calculated according to the first sending time, the first receiving time, the second sending time and the second receiving time. In this embodiment, let the first sending time be Ta1, the first receiving time be Tb1, the second sending time be Tb2, the second receiving time be Ta2, let the first clock difference between the cloud mobile phone and the cloud mobile phone server be deltaT1, let the time spent for one round of transmission of the data packet between the cloud mobile phone end and the synchronization server be RTT1, and the first clock difference be deltaT1, then RTT 1= (Tb 1-Ta 1) + (Ta 2-Tb 2), deltaT1 = Ta 1+ (RTT 1/2) -Tb1.
The cloud mobile phone server controls the cloud mobile phone to send a second test data packet to the corresponding shooting equipment and controls the cloud mobile phone to receive a second reply data packet sent by the corresponding shooting equipment according to the second test data packet, and calculates the time difference between the shooting equipment and the corresponding cloud mobile phone according to the sending time and the receiving time of the second test data packet and the second reply data packet and marks the time difference as the second clock difference. The method comprises the following steps: the cloud mobile phone server controls the cloud mobile phone to send a second test data packet to the corresponding shooting device and controls the cloud mobile phone to receive a second reply data packet sent by the corresponding shooting device according to the second test data packet, records the time of the cloud mobile phone to send the second test data packet as third sending time, the time of the corresponding shooting device to receive the second test data packet as third receiving time, records the time of the corresponding shooting device to send the second reply data packet as fourth sending time, records the time of the cloud mobile phone to receive the second reply data packet as fourth receiving time, and calculates a second clock difference value between the shooting device and the corresponding cloud mobile phone according to the third sending time, the third receiving time, the fourth sending time and the fourth receiving time. Let the third sending time be Tc1, the third receiving time be Td1, the fourth sending time be Td2, the fourth receiving time be Tc2, the time spent for one round of transmission of the data packet between the cloud mobile phone and the photographing device (client) be RTT2, the second clock difference be RTT 2= (Td 1-Tc 1) + (Tc 2-Td 2), then RTT 2= (Td 1-Tc 1) + (Tc 2-Td 2); deltat2=tc1+ (RTT 2/2) -Td1.
And calculating a preset clock correction value, and calculating the time difference between the shooting equipment and the cloud mobile phone server according to the first clock difference value and the second clock difference value and recording the time difference as the preset clock correction value. And obtaining a time difference value between the shooting equipment and the cloud mobile phone server according to the calculation, namely, a preset clock correction value, wherein the preset clock correction value=the first clock difference value+the second clock difference value. Cloud mobile phone server time = shooting device time value + preset clock correction value.
And synchronizing the audio and video data, wherein the cloud mobile phone server controls each cloud mobile phone to convert the first time stamp information into second time stamp information taking the time of the cloud mobile phone server as a reference according to the preset clock correction value according to the multipath audio and video synchronization instruction, and the time in the first time stamp information plus the preset clock correction value is the time taking the cloud mobile phone server as the reference, so that the second time stamp information is obtained. The method comprises the following specific steps: the cloud mobile phone server controls each cloud mobile phone to put received audio and video data into a buffer queue, converts first time stamp information of a head information area of each frame of data packet in the audio and video data into second time stamp information taking the cloud mobile phone server as a reference, and reports the second time stamp information to the cloud mobile phone server, the cloud mobile phone receives the second time stamp information reported by each cloud mobile phone and then puts the second time stamp information into a preset time stamp queue, a timer is arranged in the cloud mobile phone server, a plurality of time stamp information in a time period can be selected from the preset time stamp queue every other fixed time, the time stamp information is sent to a plurality of cloud mobile phones, and after the cloud mobile phones receive the time stamp information from the cloud mobile phone server, the cloud mobile phones screen audio and video data matched with the time stamp information from the buffer queue of the cloud mobile phone server, so that the cloud mobile phones are triggered at fixed time intervals, the audio and video data are matched with the time stamp, the audio and video synchronization effect is achieved, the audio and video synchronization problem that the audio and video synchronization effect is not shot from a camera-side to a plurality of camera-phone network is overcome.
And sending the free view live broadcast instruction, and the cloud live broadcast control console acquires the free view live broadcast instruction input by the terminal equipment and sends the live broadcast view instruction to the cloud mobile phone server.
And receiving a free view live broadcast instruction, wherein the cloud mobile phone server receives a live broadcast view instruction sent by a cloud live broadcast control console, and the free view live broadcast instruction is input to the cloud live broadcast control console through a terminal device by guide broadcast.
And the cloud mobile phone server controls the cloud mobile phone to reconstruct images of video data in the audio and video data uploaded by all shooting devices at the same moment according to the free view live broadcast instruction to obtain video data containing virtual view points, and the video data containing the virtual view points after image reconstruction is sent to the push-pull streaming server and pushed to a single live broadcast window in a live broadcast platform by the push-pull streaming server to play the video of the free view angles. In this embodiment, the freeview live broadcast instruction is a full-range freeview instruction or a wonderful moment freeview instruction, where the full-range freeview instruction is a freeview live broadcast of video data in audio and video data corresponding to any moment, the wonderful moment freeview instruction is a time point when the wonderful moment is taken as the free view play required, and the wonderful moment is a moment when the director or director considers that the shooting object is more wonderful in the live broadcast process. When the free view live broadcast instruction is a whole-course free view instruction, the free view live broadcast specifically comprises: the cloud mobile phone server controls the cloud mobile phone to send all audio and video data to a preset image processing server, and image reconstruction is carried out on images in video data uploaded by all shooting devices at each moment according to a preset image reconstruction algorithm, so that video data containing virtual viewpoints are obtained, the video data after image reconstruction at each moment is sent to a push-pull streaming server, and the push-pull streaming server pushes video data to a live broadcasting platform for video playing at free viewpoints. When the free view live broadcast instruction is a wonderful moment free view instruction, the free view live broadcast is specifically: the cloud mobile phone server controls the cloud mobile phone to send all audio and video data corresponding to the wonderful time to the preset image processing server, the preset image processor carries out image reconstruction on images in the audio and video data corresponding to the wonderful time according to a preset image reconstruction algorithm to obtain video data containing virtual viewpoints and recorded as the wonderful time video data, and the wonderful time video data is sent to the cloud broadcasting guide platform by the preset image processor and pushed to the live broadcasting platform by the cloud broadcasting guide platform according to the corresponding time points for playing.
In this embodiment, the platform for controlling the live view angle by using the cloud director as the director may be set in any mobile phone or other electronic terminals, and the cloud director performs the following steps:
receiving the video data at the wonderful moment, and the cloud broadcasting guide station receives and stores the video data at the wonderful moment sent by the preset image processor;
pushing the video data at the wonderful moment, and sending the video data at the wonderful moment to a push-pull streaming server by a cloud broadcasting guide platform according to the time point corresponding to the wonderful moment, and pushing the video data to a live broadcast platform by the push-pull streaming server for broadcasting.
In this embodiment, the live video may be watched by different viewers, specifically, the viewers log in to a designated live video platform through their own terminal devices (mobile phone or tablet computer or television or computer) and send a live video request to the live video platform, where the live video request includes a viewer address, and the viewer address is identity information of the terminal devices of the viewers and is used to distinguish different viewers, and the live video platform establishes connection with a push-pull stream server according to the live video request and a pre-stored push-pull stream address, and receives video data after image reconstruction sent by the push-pull stream server. And a viewer can rotate the live video by clicking different virtual points in the live video watching process, so that the single live window can watch shooting objects at different visual angles.
According to the video live broadcasting method of the free view angle, shooting of shooting equipment is controlled through the cloud live broadcasting console according to the unified remote control instruction input by the live broadcasting, the operation flow of the shooting equipment in video live broadcasting is simplified, in addition, audio and video data shot by the shooting equipment are uploaded to the cloud live broadcasting console, and the audio and video data are allowed to be uploaded to one cloud mobile phone, so that a shooting manager can conveniently monitor shooting working conditions through low-definition streams forwarded by the cloud mobile phone; the cloud mobile phone server receives the cloud live control console installed in the broadcasting-guiding terminal equipment and the broadcasting-guiding platform to control the live process, so that the broadcasting in the whole video live can be remotely controlled in any place, the trouble that the video live is limited by the condition of a live field is eliminated, the efficiency of the video live is improved, and in addition, video data containing virtual viewpoints after image reconstruction is sent to the push-pull streaming server and pushed to the live platform for video playing at the free viewing angles by the push-pull streaming server, so that a viewer can watch live pictures at different viewing angles in a single window, and the experience of the viewer is enhanced.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way; those skilled in the art can smoothly practice the invention as shown in the drawings and described above; however, those skilled in the art will appreciate that many modifications, adaptations, and variations of the present invention are possible in light of the above teachings without departing from the scope of the invention; meanwhile, any equivalent changes, modifications and evolution of the above embodiments according to the essential technology of the present invention still fall within the scope of the present invention.
Claims (6)
1. A video live broadcast method of free view angle is characterized in that: the method is applied to a cloud live broadcast service system, the cloud live broadcast service system comprises a cloud mobile phone server, a cloud live broadcast control console, a push-pull stream server, a cloud broadcast guiding console and shooting equipment which is arranged at different view angles corresponding to shooting objects and has included angles between adjacent shooting equipment within a preset included angle range, the cloud mobile phone server is provided with cloud mobile phones with the same quantity as the shooting equipment, each cloud mobile phone is connected with only the shooting equipment, and the method comprises the following steps:
receiving audio and video data, wherein a cloud mobile phone server controls each cloud mobile phone to receive audio and video data packets of corresponding visual angles of shooting objects uploaded by corresponding shooting equipment and store the audio and video data packets in corresponding buffer queues, each cloud mobile phone receives audio and video data packets which contain first timestamp information based on the time of the shooting equipment, and the audio and video data are audio and video shooting performed by shooting equipment arranged at positions of the shooting objects corresponding to different visual angles according to unified remote control instructions input by a direct broadcasting console and are uploaded to the corresponding cloud mobile phones;
synchronizing the audio and video stream data, and controlling each cloud mobile phone to convert the first time stamp information into second time stamp information taking the time of the cloud mobile phone server as a reference by the cloud mobile phone server according to a multi-channel audio and video synchronization instruction by the cloud mobile phone server;
the method further comprises the step of generating a preset clock correction value before synchronizing the audio and video stream data, wherein the step of generating the preset clock correction value specifically comprises the following steps:
calculating a first clock difference value, wherein the cloud mobile phone server sends a first test data packet to the cloud mobile phone, receives a first reply data packet sent by the cloud mobile phone according to the first test data packet, calculates a time difference value between the cloud mobile phone and the cloud mobile phone server according to the sending time and the receiving time of the first test data packet and the first reply data packet, and records the time difference value as the first clock difference value;
calculating a second clock difference value, wherein the cloud mobile phone server controls the cloud mobile phone to send a second test data packet to the corresponding shooting equipment and controls the cloud mobile phone to receive a second reply data packet sent by the corresponding shooting equipment according to the second test data packet, calculates the time difference value between the shooting equipment and the corresponding cloud mobile phone according to the sending time and the receiving time of the second test data packet and the second reply data packet, and marks the time difference value as the second clock difference value;
calculating a preset clock correction value, calculating a time difference between the shooting equipment and the cloud mobile phone server according to the first clock difference value and the second clock difference value, and recording the time difference as the preset clock correction value;
receiving a free view live broadcast instruction, wherein a cloud mobile phone server receives a live broadcast view instruction sent by a cloud live broadcast control console, and the free view live broadcast instruction is input to the cloud live broadcast control console through a terminal device by a guide broadcast;
when the free view live broadcast instruction is a highlight free view instruction, the highlight free view instruction is a time point when the highlight free view instruction is to be played in the free view, the cloud mobile phone server controls the cloud mobile phone to send all audio and video data corresponding to the highlight to the preset image processing server, the preset image processor carries out image reconstruction on images in the audio and video data corresponding to the highlight according to a preset image reconstruction algorithm to obtain video data containing a virtual viewpoint and recorded as highlight video data, and the highlight video data is sent to the cloud broadcasting guide platform by the preset image processor and pushed to the live broadcast platform by the cloud broadcasting guide platform according to the corresponding time point for playing;
the cloud live broadcast control console controls shooting equipment arranged at the corresponding position of different visual angles of a shooting object to shoot audio and video according to a remote control instruction input by the guide broadcast, and uploads the shot audio and video data to a corresponding cloud mobile phone;
the method comprises the steps that a multipath audio and video synchronization instruction is sent, and a cloud live broadcast control console obtains the multipath audio and video synchronization instruction input by a terminal device in a guide mode and sends the multipath audio and video synchronization instruction to a cloud mobile phone server;
a free view live broadcast instruction is sent, and a cloud live broadcast control console acquires the free view live broadcast instruction input by a terminal device in a guided broadcast mode and sends the live broadcast view instruction to a cloud mobile phone server;
receiving the video data at the wonderful moment, and the cloud broadcasting guide station receives and stores the video data at the wonderful moment sent by the preset image processor;
and pushing the video data at the wonderful moment, and sending the video data at the wonderful moment to a push-pull streaming server by the cloud broadcasting guide platform according to the time point corresponding to the wonderful moment, and pushing the video data to a live broadcasting platform by the push-pull streaming server for broadcasting.
2. The video live broadcast method of claim 1, wherein: when the free view live broadcast instruction is a whole free view instruction, the whole free view instruction is that video data in audio and video data corresponding to any moment needs free view live broadcast, and the free view live broadcast specifically includes: the cloud mobile phone server controls the cloud mobile phone to send all audio and video data to a preset image processing server, and image reconstruction is carried out on images in video data uploaded by all shooting devices at each moment according to a preset image reconstruction algorithm, so that video data containing virtual viewpoints are obtained, the video data after image reconstruction at each moment is sent to a push-pull streaming server, and the push-pull streaming server pushes video data to a live broadcasting platform for video playing at free viewpoints.
3. The video live broadcast method of claim 1, wherein: the video data after image reconstruction is sent to a push-pull streaming server and pushed to a live broadcast platform by the push-pull streaming server to play the video with free view angle specifically comprises the following steps: and sending the video data subjected to image reconstruction to a push-pull streaming server, pushing the video data to a corresponding live broadcast platform by the push-pull streaming server according to live broadcast requests of different viewer addresses to play video at a free view angle, and sending the live broadcast requests after the live broadcast platform establishes connection with the push-pull streaming server according to the pre-stored push-pull streaming addresses.
4. The video live broadcast method of claim 1, wherein: and the live broadcast platform plays the video with the free view angle, and a viewer logs in through a mobile phone terminal or a computer or a television to watch the video.
5. An electronic device, comprising: a processor;
a memory; and a program, wherein the program is stored in the memory and configured to be executed by the processor, the program comprising means for performing a live video method of one of the freeview angles of any of claims 1-4.
6. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program is executed by a processor to perform a live video method of a freeview as claimed in any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110546103.8A CN113542896B (en) | 2021-05-19 | 2021-05-19 | Video live broadcast method, equipment and medium of free view angle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110546103.8A CN113542896B (en) | 2021-05-19 | 2021-05-19 | Video live broadcast method, equipment and medium of free view angle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113542896A CN113542896A (en) | 2021-10-22 |
| CN113542896B true CN113542896B (en) | 2024-02-23 |
Family
ID=78094691
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110546103.8A Active CN113542896B (en) | 2021-05-19 | 2021-05-19 | Video live broadcast method, equipment and medium of free view angle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113542896B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114501044B (en) * | 2021-12-23 | 2023-12-08 | 百果园技术(新加坡)有限公司 | Live broadcast data processing method, device, equipment and storage medium |
| CN114827647B (en) * | 2022-04-15 | 2024-03-19 | 北京百度网讯科技有限公司 | Live broadcast data generation method, device, equipment, medium and program product |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106550239A (en) * | 2015-09-22 | 2017-03-29 | 北京同步科技有限公司 | 360 degree of panoramic video live broadcast systems and its implementation |
| CN106921866A (en) * | 2017-05-03 | 2017-07-04 | 广州华多网络科技有限公司 | The live many video guide's methods and apparatus of auxiliary |
| CN107396133A (en) * | 2017-07-20 | 2017-11-24 | 深圳市佳创视讯技术股份有限公司 | Free viewpoint video director method and system |
| CN108924526A (en) * | 2017-03-27 | 2018-11-30 | 华为软件技术有限公司 | Video broadcasting method, terminal and system |
| JP2019012533A (en) * | 2018-08-08 | 2019-01-24 | キヤノン株式会社 | Information processing apparatus, information processing method, and program |
| CN111355966A (en) * | 2020-03-05 | 2020-06-30 | 上海乐杉信息技术有限公司 | Surrounding free visual angle live broadcast method and system |
| CN111669564A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Image reconstruction method, system, device and computer readable storage medium |
| CN111787286A (en) * | 2020-07-22 | 2020-10-16 | 杭州当虹科技股份有限公司 | Method for realizing multichannel synchronous recording system |
| CN111917988A (en) * | 2020-08-28 | 2020-11-10 | 长沙摩智云计算机科技有限公司 | Remote camera application method, system and medium of cloud mobile phone |
-
2021
- 2021-05-19 CN CN202110546103.8A patent/CN113542896B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106550239A (en) * | 2015-09-22 | 2017-03-29 | 北京同步科技有限公司 | 360 degree of panoramic video live broadcast systems and its implementation |
| CN108924526A (en) * | 2017-03-27 | 2018-11-30 | 华为软件技术有限公司 | Video broadcasting method, terminal and system |
| CN106921866A (en) * | 2017-05-03 | 2017-07-04 | 广州华多网络科技有限公司 | The live many video guide's methods and apparatus of auxiliary |
| CN107396133A (en) * | 2017-07-20 | 2017-11-24 | 深圳市佳创视讯技术股份有限公司 | Free viewpoint video director method and system |
| JP2019012533A (en) * | 2018-08-08 | 2019-01-24 | キヤノン株式会社 | Information processing apparatus, information processing method, and program |
| CN111669564A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Image reconstruction method, system, device and computer readable storage medium |
| CN111355966A (en) * | 2020-03-05 | 2020-06-30 | 上海乐杉信息技术有限公司 | Surrounding free visual angle live broadcast method and system |
| CN111787286A (en) * | 2020-07-22 | 2020-10-16 | 杭州当虹科技股份有限公司 | Method for realizing multichannel synchronous recording system |
| CN111917988A (en) * | 2020-08-28 | 2020-11-10 | 长沙摩智云计算机科技有限公司 | Remote camera application method, system and medium of cloud mobile phone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113542896A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180077430A1 (en) | Cloned Video Streaming | |
| US11153615B2 (en) | Method and apparatus for streaming panoramic video | |
| US20150208103A1 (en) | System and Method for Enabling User Control of Live Video Stream(s) | |
| CN113542896B (en) | Video live broadcast method, equipment and medium of free view angle | |
| US20150222815A1 (en) | Aligning videos representing different viewpoints | |
| KR101934200B1 (en) | Method and system for media synchronization | |
| CN105635675B (en) | A kind of panorama playing method and device | |
| KR101446995B1 (en) | Helmet for imaging multi angle video and method thereof | |
| CN113542897A (en) | Audio and video live broadcast method suitable for multi-view live broadcast | |
| KR102025157B1 (en) | System and method for transmitting a plurality of video image | |
| TW201806380A (en) | Video splitting device and monitoring method | |
| CN108418832A (en) | A kind of virtual reality shopping guide method, system and storage medium | |
| CN108989833B (en) | Method and device for generating video cover image | |
| CN113572975A (en) | Video playing method, device and system and computer storage medium | |
| CN117221626B (en) | Video data processing method and device | |
| CN109788366A (en) | A kind of 3D interaction live broadcast system | |
| CN110392275B (en) | Sharing method and device for manuscript demonstration and video networking soft terminal | |
| CN106851354B (en) | Method and related device for synchronously playing recorded multimedia in different places | |
| KR20150030889A (en) | Method and apparatus for providing multi angle video broadcasting service | |
| US20150150040A1 (en) | Interactive audio/video broadcast system, method for operating the same and user device for operation in the interactive audio/video broadcast system | |
| CN115529298B (en) | System, method and device for transmitting dense video and audio | |
| KR102637147B1 (en) | Vertical mode streaming method, and portable vertical mode streaming system | |
| CN108881810A (en) | The method of transmitting audio-video stream | |
| RU136663U1 (en) | REMOTE VIEWING SYSTEM OR SIMILAR VIEW | |
| WO2022222533A1 (en) | Video playing method, apparatus and system, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |