WO2021164352A1 - 一种网络直播数据的管理方法以及相关装置 - Google Patents
一种网络直播数据的管理方法以及相关装置 Download PDFInfo
- Publication number
- WO2021164352A1 WO2021164352A1 PCT/CN2020/130005 CN2020130005W WO2021164352A1 WO 2021164352 A1 WO2021164352 A1 WO 2021164352A1 CN 2020130005 W CN2020130005 W CN 2020130005W WO 2021164352 A1 WO2021164352 A1 WO 2021164352A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- streaming
- push
- media content
- live broadcast
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 134
- 238000013523 data management Methods 0.000 title abstract description 15
- 230000006978 adaptation Effects 0.000 claims description 35
- 238000007726 management method Methods 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 3
- 230000005856 abnormality Effects 0.000 claims 1
- 230000008569 process Effects 0.000 abstract description 67
- 238000012545 processing Methods 0.000 description 32
- 238000010586 diagram Methods 0.000 description 25
- 230000006870 function Effects 0.000 description 15
- 230000003993 interaction Effects 0.000 description 13
- 230000005540 biological transmission Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 230000010354 integration Effects 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000001934 delay Effects 0.000 description 4
- 230000011664 signaling Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 238000012384 transportation and delivery Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009527 percussion Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2347—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving video stream encryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
- H04N21/26216—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints involving the channel capacity, e.g. network bandwidth
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/26613—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel for generating or managing keys in general
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/27—Server based end-user applications
- H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
- H04N21/2743—Video hosting of uploaded data from client
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
Definitions
- This application relates to the field of Internet technology, in particular to the management technology of webcast data.
- webcast rooms have attracted more and more users with novel forms and rich contents.
- different webcast rooms are used to play different live content, and the terminal can correspondingly meet the different viewing needs of users by accessing different webcast rooms.
- the interactive process between the terminal and other terminals can be video streaming, for example: the current multi-person audio and video call content is live streaming, and after transcoding or multi-stream mixing, it is distributed to the content Distribution network (Content Delivery Network, CDN), and then other users can stream and watch through the information transmission protocol.
- content Distribution network Content Delivery Network, CDN
- the present application provides a method and related device for managing webcast data, which can effectively avoid webcast delays caused by parameter differences between multiple media contents or network environment differences, and improve the stability of the webcast process. .
- the first aspect of this application provides a method for managing webcast data, which is executed by a multipoint control device, and the method specifically includes:
- the streaming request includes streaming parameters, the streaming parameters being determined by the data access device based on the media content characteristics of the streaming data to be sent, the streaming Streaming data includes media content data from at least two data sources;
- the second aspect of the present application provides another data management method, which is executed by a data access device, and the method specifically includes:
- control server specifically includes:
- the scheduling module is configured to receive a streaming request sent by a data access device, where the streaming request includes streaming parameters, and the streaming parameters are determined by the data access device based on the media content characteristics of the streaming data to be sent Yes, the push streaming data includes media content data from at least two data sources;
- An adaptation module configured to determine a target port adapted to the characteristics of the media content according to the streaming parameters
- a media content module configured to receive the push data sent by the data access device through the target port
- the push streaming module is configured to push the push streaming data to the live broadcast device, so that the live broadcast device plays the media content data during the web live broadcast.
- the fourth aspect of the present application provides a network device, which can be applied to a webcast system, and the network device specifically includes:
- the access module is used to receive push data sent by at least two data sources;
- the streaming control module is configured to determine corresponding streaming parameters according to the media content characteristics of the streaming data
- the streaming control module is further configured to send the streaming parameter to the multipoint control device, where the streaming parameter is used to instruct the multipoint control device to determine a target port adapted to the media content feature;
- the flow control module is further configured to receive target port information sent by the multipoint control device;
- the access module is further configured to send the push data through the target port.
- a fifth aspect of the present application provides a computer device, including: a memory, a processor, and a bus system; the memory is used to store program code; the processor is used to execute the above-mentioned first aspect according to instructions in the program code The method for managing webcast data or the method for managing webcast data described in the second aspect above.
- a sixth aspect of the present application provides a computer-readable storage medium having instructions stored in the computer-readable storage medium, which when run on a computer, cause the computer to execute the method for managing webcast data described in the first aspect above , Or the method for managing webcast data as described in the second aspect above.
- the seventh aspect of the present application provides a computer program product, including instructions, which when run on a computer, cause the computer to execute the method for managing webcast data described in the first aspect, or the webcast described in the second aspect above Data management methods.
- the multipoint control device receives the push streaming request sent by the data access device, and determines the push streaming parameters set based on the media content characteristics of the push streaming data from multiple data sources in the data access device; then, the multipoint control device The target port is determined according to the push streaming parameters, and the target port can be adapted to the media content characteristics of the push streaming data; further, the multipoint control device receives the push streaming data sent by the data access device through the target port, and pushes the push streaming to the live broadcast device. Streaming data, so that the live broadcast device can play the media content data in the push stream data during the webcasting process. .
- Figure 1 is a network architecture diagram of the operation of the data management system
- Figure 2 is a system architecture diagram of a webcast provided by an embodiment of the application
- FIG. 3 is a flowchart of a method for managing webcast data provided by an embodiment of the application
- FIG. 5 is a schematic structural diagram of a control server provided by an embodiment of this application.
- FIG. 6 is a schematic structural diagram of a network device provided by an embodiment of this application.
- Fig. 7 is a system framework diagram of another webcast provided by an embodiment of the application.
- FIG. 8 is a schematic diagram of an interaction of load balancing provided by an embodiment of this application.
- FIG. 9 is a schematic diagram of interaction of multi-terminal media content mixed streaming provided by an embodiment of this application.
- FIG. 10 is a schematic diagram of interaction for pushing media content on multiple terminals according to an embodiment of this application.
- FIG. 11 is a schematic diagram of another multi-terminal media content push interaction diagram provided by an embodiment of this application.
- FIG. 12 is a flowchart of another method for managing webcast data according to an embodiment of the application.
- FIG. 13 is a schematic diagram of updating a webcast page provided by an embodiment of the application.
- FIG. 14 is a schematic structural diagram of another terminal device provided by an embodiment of this application.
- FIG. 15 is a schematic structural diagram of another control server provided by an embodiment of this application.
- Multipoint Control Unit In this application, it is also called a multipoint control device. It is the core part of the audio and video conference system. The main function is to provide users with group meetings and multi-group meetings. service.
- SFU Selective Forwarding Unit It is simply used to forward audio and video data, and does not involve the conversion and merging of audio and video data formats, parameters, etc. The conversion and merging of audio and video data formats, parameters, etc. are left to the viewer Processing after receiving the data.
- SDK Software Development Kit
- Mixing It is a way of controlling, synchronizing and mixing video and audio data streams, that is, combining video and audio data.
- Push Streaming It is the process of transmitting the encapsulated content data in the collection stage to the designated device.
- the data management method provided in this application can be applied to a system or program that supports the webcast function, such as a live broadcast platform.
- the system that supports the webcast function can run in the network architecture as shown in FIG. 1.
- FIG. 1 it is a network architecture diagram of the system operation.
- Figure 1 shows that the system can interact with multiple terminal devices for media content.
- the terminal establishes a connection with the server through the network, and the server uploads the audio and video streams to the terminal. Repost and push the stream to the target terminal to realize the webcast; it is understandable that a variety of terminal devices are shown in Figure 1.
- more or fewer types of terminal devices can participate in the webcast process.
- the specific number and type depends on the actual scenario, and it is not limited here.
- one server is shown in Figure 1, but in the actual scenario, multiple servers can also participate, especially in multi-content application interaction In the scenario, the number of specific servers depends on the actual scenario.
- the above-mentioned webcast data management system can be run on the server in the form of a program, can also be run as a system component in the server, or as a kind of cloud service program.
- the specific operation mode depends on It depends on the actual scene and is not limited here.
- embodiments of the present application provide a method for managing webcast data. This method is aimed at webcasting of multi-terminal media content integration. Targeted port selection for the media content of multiple terminals, so that the media content of multiple terminals is better integrated and adapted to the live broadcast equipment, which reduces the live broadcast delay caused by network fluctuations and avoids accidental conditions. The live streaming is blocked, which improves the stability of the webcasting process.
- Interface machine used to transfer audio and video data sent by terminal equipment and signaling in the audio and video process.
- Flow control machine used for real-time adjustment of various audio and video parameters and audio routing in the audio and video process.
- Configuration Center Contains the location distribution and real-time load status of all machines, and can synchronize the location distribution and real-time load status of all machines to the information center.
- Policy scheduling server responsible for services such as request authentication, signature encryption, and security confirmation. According to the user's location information, the location of the machine, and the load information, the interface machine and flow control machine are allocated nearby, so that users of the same room and the same operator can gather as much as possible to the same interface machine.
- Query service including but not limited to querying user information, relationship chain, remarks, grouping, group relationship chain, user-generated information, etc.
- Login service a universal account login component for quickly completing account integration and access to audio and video or instant messaging cloud services.
- Signaling server Mainly responsible for the access and login of web-side users, the transparent transmission of signaling, the allocation of relay servers, the negotiation and delivery of audio and video initial parameters, the negotiation of connection channels, and the negotiation and delivery of security parameters.
- Transfer server First, it is used to simulate web instant messaging for web users to establish a direct connection with the web, including the establishment of interactive connections, the negotiation of the security protocol key of the data packet transmission layer, the transfer of audio and video data based on the secure real-time transmission protocol, and Real-time Control Protocol (RTCP) audio and video quality control. Secondly, it is used for simulating SDK users facing the interface machine to establish a connection with the existing interactive live broadcast system, including entering the room, data transmission, etc. Thirdly, it is used to connect web users and SDK users' room entry and exit, status notification and other signaling, and then to link the audio and video data transcoding and transpackaging of both web users and SDK users in exchange for a unified identification ID, compatible Room management and account system for interactive live broadcast.
- RTCP Real-time Control Protocol
- the method for managing webcast data involves the interaction process between the interface machine, the multipoint control device, and the relay server in the live broadcast device, which is equivalent to adding multiple points between the interface machine and the relay server. Control equipment to manage media data during the webcast.
- the apparatus for managing live webcast data receives a streaming request sent by a data access device through a multipoint control device, and determines the media content characteristics based on the streaming data from multiple data sources in the data access device The set push streaming parameters; then, the multipoint control device determines the target port according to the push streaming parameters, and the target port can be adapted to the media content characteristics of the push streaming data; further, the multipoint control device receives data access through the target port Push streaming data sent by the device and push the push streaming data to the live broadcast device, so that the live broadcast device plays the media content of the push streaming data during the web live broadcast process.
- the process of converging media content from multiple terminals and performing webcasting is realized. Because the above method selects specific ports for media content from multiple data sources during the streaming process, it can make the media content from multiple data sources very good. Good integration and adaptation to live broadcast equipment, reducing live broadcast delays caused by network fluctuations, and avoiding live streaming blockage caused by accidental conditions, and improving the stability of the web live broadcast process.
- FIG. 3 is a flowchart of a method for managing webcast data according to an embodiment of the application.
- the method is applied to a webcast system.
- the webcast system includes a data access device, a multipoint control device, and a live broadcast.
- the embodiments of this application at least include the following steps:
- the data access device determines push streaming parameters according to the media content characteristics of the push streaming data.
- the push streaming data includes media content data from at least two data sources.
- the push streaming data may be media content data of a three-person video call; and the push streaming parameters are parameters determined based on the characteristics of the media content data. For example: audio format, number of channels, audio bit rate, etc., or video encoding format, allocation rate, bit rate, multi-channel picture layout and other parameters.
- the data source can be a terminal device used to upload media content data such as audio and video in real time, such as a mobile phone that uploads recorded videos in real time; the data source can also be a platform that provides data resources on the network, such as video content. Many different websites, the specific format depends on the actual scene.
- the specific push streaming parameters can be one or a combination of multiple parameters in the above examples, and can be set according to the application scenario of the push streaming data, for example: if the push streaming data is for a multi-person audio call For audio content data, the streaming parameters should be the audio format, the number of channels, and the audio bit rate; the specific parameter format depends on the actual scene and is not limited here.
- the process of determining the push streaming parameters can be implemented based on an artificial intelligence model, that is, once the push streaming data is accessed, various parameters in the push streaming data can be extracted through a preset analysis model, for example, assume that the network The demand for live broadcast is high sound quality, and the streaming parameters analyzed through the analysis model should include the audio format, the number of channels, the audio bit rate, and extract specific data.
- the data access device sends a streaming request to the multipoint control device.
- the push streaming request includes push streaming parameters.
- the multipoint control device receives the streaming request and obtains streaming parameters.
- the multipoint control device determines a target port adapted to the characteristics of the media content according to the streaming parameters.
- the multipoint control device can allocate data adaptation devices, audio processing devices, video processing devices, and streaming devices according to the global load information and the location information of the request source, and send port allocation requests to these devices.
- the configuration equipment, audio processing equipment, video processing equipment and streaming equipment establish a session locally and allocate target ports.
- the multipoint control device is associated with multiple data adaptation devices, audio processing devices, video processing devices, and streaming devices in the webcast system.
- the multipoint control device collects the information of the associated devices. Load status, which can specifically include CPU usage, number of CPU cores, inflow, outflow, inflow, outflow, etc., and then select the corresponding device according to preset conditions, for example: the required associated device is indicated in the push flow parameter If the CPU usage rate is less than 50%, the multipoint control device can traverse the associated devices to find devices with a CPU usage rate less than 50%, and determine the address of the device and the corresponding port.
- the above-mentioned associated device may also be a device integrated in a multipoint control device, and the specific device form depends on the actual scenario and is not limited here.
- the network fluency in the webcasting process can be guaranteed, that is, the corresponding target port is selected according to the media content parameters required by different terminals, and the process of multi-terminal media content integration and streaming is realized based on the target port , which can ensure the smoothness of the network during the webcast.
- the multipoint control device sends target port information to the data access device.
- the multipoint control device determines the corresponding port identifier and device address information corresponding to the port according to the target port determined in step 304, packs these information into target port information and sends it to the data access device.
- the multipoint control device and the data access device configure the target port synchronously to establish a connection.
- the process of synchronously configuring the target port is the process of determining the corresponding device according to the device address information and selecting the port according to the port identifier. After the device and the port are determined, the connection can be established.
- the data access device sends the push stream data to the multipoint control device.
- the data access device based on the port connection established in step 306, the data access device sends push data to the multipoint control device.
- the multipoint control device sends the push streaming data to the live broadcast device.
- the media content type of the push streaming data such as audio or video
- the push streaming data is classified according to the media content type to determine the push streaming The classification type of the data; further, the push streaming data is respectively converted into a target format corresponding to the classification type, and pushed to the live broadcast device. For example, if the audio data format required by the target format is MP3 and the video data format is FLV, the push stream data is classified into audio data and video data, and then converted to the target format respectively.
- the process of media content classification can be based on voice recognition, that is, the audio part of the push streaming data can be automatically identified and switched to the corresponding format, thereby ensuring the pertinence of the push streaming process.
- the live broadcast device plays the media content according to the push streaming data.
- media content data from at least two data sources are integrated into the push data pushed to the live broadcast device.
- the media content data can be parallel or stacked, and can also respond to The user switches the media content accordingly to the operation triggered by the play page associated with the live broadcast device. For example, if the live broadcast device is associated with the H5 page, the live broadcast device can play the corresponding media content according to the user's selection of media content on the H5 page.
- the push streaming request sent by the data access device is received by the multipoint control device, and the push streaming parameters set based on the media content characteristics of the push streaming data from multiple data sources in the data access device are determined; Then, the multipoint control device determines the target port according to the push streaming parameters, and the target port can be adapted to the media content characteristics of the push streaming data; further, the multipoint control device receives the push streaming data sent by the data access device through the target port, And push the push streaming data to the live broadcast device, so that the live broadcast device plays the media content of the push streaming data during the web live broadcast process. In this way, the process of integrating media content from multiple data sources and performing live webcasting is realized.
- the above method selects specific ports for media content from multiple data sources during the streaming process, it can make media from multiple data sources
- the content is well integrated and adapted to the live broadcast equipment, reducing the live broadcast delay caused by network fluctuations, and avoiding the live streaming blockage caused by accidental conditions, and improving the stability of the web live broadcast process.
- FIG. 4 is a process of another method for managing webcast data provided by an embodiment of the application.
- the figure is applied to a webcast system.
- the webcast system includes a data access device, a multipoint control device, and a live broadcast device.
- the embodiment of the present application includes at least the following steps:
- the multipoint control device periodically performs load balancing.
- the process of load balancing performed by the multipoint control device is to obtain the load information of its associated device according to a preset period, and the load information is used to indicate the abnormal condition of the associated device; if the load information does not meet the preset condition, then Determine the abnormal associated device based on the load information; then switch the address of the abnormal associated device so that the load information of the associated device meets the preset condition.
- the preset condition is that the response delay between the multipoint control device and the associated device is less than 10 milliseconds, the associated device with the response delay less than 10 milliseconds should be detected.
- the associated device can be a device that is associated with the multipoint control device through the network and can achieve corresponding functions; the associated device can also be multiple devices integrated locally in the multipoint control device.
- the specific association form depends on the actual scenario. Depends.
- the data access device sends a mixed stream request to the multipoint control device.
- the media content in the general webcast is a combined content of video and audio
- a mixing process is required.
- the multipoint control device determines a set of free ports.
- the set of free ports of the multipoint control device namely the available associated devices and the ports on the available associated devices can be determined during the process of mixing.
- the data access device determines streaming parameters according to the streaming data.
- the data access device sends a streaming request to the multipoint control device.
- the process of sending the streaming request can reuse the session of the streaming process, and if the user needs both streaming and streaming, the two requests can be combined into one request.
- the multipoint control device receives the streaming request and obtains streaming parameters.
- the multipoint control device determines an adapted target port according to the streaming parameter.
- the selection of the target port can be performed based on the set of free ports determined during the flow mixing process, which reduces the range of port judgment and improves the efficiency of target port determination.
- the multipoint control device sends target port information to the data access device.
- the multipoint control device synchronizes the target port with the data access device to establish a connection.
- the data access device obtains the user type of the data source to determine the corresponding target port.
- the data access device sends the push stream data to the multipoint control device.
- step 408 to step 411 are similar to step 305 to step 308 in the embodiment described in FIG. 3, and related features can be referred to, and will not be repeated here.
- the multipoint control device adjusts the push data to the target format and target page layout.
- the target format and target page layout may be sent by the live broadcast device, or recorded in the historical data processing record of the multipoint control device. If the target format and target page layout are sent by the live broadcast device, the multipoint control device needs to receive the target parameter sent by the live broadcast device. The target parameter is determined based on the characteristics of the media content during the webcast, and then pushes the stream according to the target parameter The data is adjusted.
- the target format and target page layout can also be set by the terminal device associated with the data access device.
- the multipoint control device can receive page layout parameters sent by the data access device. To indicate the page distribution of media content data from at least two data sources in the data access device; then adjust the push data according to the page layout parameters, such as picture-in-picture, four-square grid, nine-square grid, and other page layout methods. In this way, the media content data from multiple data sources can be flexibly integrated, and the richness of the webcast is improved.
- the target parameter can be audio, video, or a combination of audio and video; for example, in a live voice broadcast scene, the target parameter can be audio format information; in a remote desktop and other video-only scenes, the target parameter It can be video format information; in a live webcast scenario, the target parameter can include audio format information and video format information.
- the target can be audio, video, or a combination of audio and video; for example, it can achieve audio mixing and video "picture-in-picture" effects, which improves the flexibility of the live broadcast process.
- the target parameter or page layout parameter can also be analyzed based on artificial intelligence technology.
- the web live broadcast system can automatically analyze the type of push data, and when it is determined that the push data is In the case of audio data, the part corresponding to the audio in the target parameter can be automatically extracted and related settings can be made.
- the multipoint control device sends the adjusted push stream data to the live broadcast device.
- the push streaming data can be encrypted, where the encryption process can be implemented based on the target key sent by the policy scheduling device, and the target key is used to indicate the address of the live broadcast device , Set a target key for the push stream data, and then encrypt the push stream data according to the target key.
- the live broadcast device plays the media content according to the push streaming data.
- the method for managing webcast data can be implemented through multiple methods.
- Point control devices to push Real Time Messaging Protocol (rtmp) streams or HLS (HTTP Live Streaming) streams to the live broadcast system, thereby realizing simultaneous live broadcast during real-time multi-person calls.
- rtmp Real Time Messaging Protocol
- HLS HTTP Live Streaming
- the rtmp, hls and other formats can be pushed and watched to achieve a delay of 1 to 3 seconds.
- the viewing of the page can achieve a delay of 0.5 to 1 second. Due to the adaptability of the multipoint control device to the streaming process, a better live broadcast experience can be obtained in weak network scenarios, and the resolution, bit rate, frame rate, and screen layout can be flexibly customized to increase the richness of live content.
- FIG. 5 is a schematic structural diagram of a control server provided by an embodiment of the application, which is equivalent to the multipoint control device in the foregoing embodiment, and the control server 500 includes:
- the scheduling module 501 is configured to receive a streaming request sent by a data access device, the streaming request includes streaming parameters, and the streaming parameters are the media content characteristics of the data access device based on the streaming data to be sent It is determined that the push streaming data includes media content data from at least two data sources;
- the adaptation module 502 is configured to determine a target port adapted to the characteristics of the media content according to the streaming parameters;
- the media content module 503 is configured to receive the push data sent by the data access device through the target port;
- the streaming module 504 is configured to push the streaming data to the live broadcast device, so that the live broadcast device plays the media content data during the web live broadcast.
- the scheduling module 501 is further configured to obtain load information of the associated device of the multipoint control device according to a preset period, and the load information is used to indicate the Abnormal conditions of associated equipment;
- the scheduling module 501 is further configured to, if the load information does not meet a preset condition, determine the address information of an abnormal associated device based on the load information, and the preset condition is based on the relationship between the associated device and the associated device. The connection status of the multipoint control device and the load status of the associated device are determined;
- the scheduling module 501 is further configured to perform address switching of the associated device according to the address information, so that the load information of the associated device meets the preset condition.
- the adaptation module 502 is specifically configured to receive a mixed stream request sent by the data access device;
- the adaptation module 502 is specifically configured to determine a set of free ports according to the mixed stream request
- the adaptation module 502 is specifically configured to select the target port from the set of free ports according to the streaming parameter.
- the streaming module 504 is specifically configured to determine the media content type of the streaming data
- the streaming module 504 is specifically configured to classify the streaming data according to the media content type, and determine the classification type of the streaming data;
- the streaming module 504 is specifically configured to convert the streaming data into a target format corresponding to the classification type, and push it to the live broadcast device.
- the scheduling module 501 is further configured to receive target parameters sent by the live broadcast device, and the target parameters are determined based on the characteristics of the media content during the web live broadcast. of;
- the media content module 503 is further configured to adjust the push streaming data according to the target parameter.
- the scheduling module 501 is further configured to receive page layout parameters sent by the data access device, and the page layout parameters are used to instruct the data access device Page distribution of media content data from the at least two data sources in the data;
- the media content module 503 is further configured to adjust the push streaming data according to the page layout parameters.
- the webcast system further includes a policy scheduling device, and the scheduling module 501 is further configured to obtain a target key sent by the policy scheduling device, and the target key is used To indicate the address of the live broadcast device;
- the streaming module 504 is further configured to encrypt the streaming data according to the target key.
- the multipoint control device receives the push streaming request sent by the data access device, and determines the push streaming parameters set based on the media content characteristics of the push streaming data from multiple data sources in the data access device; then, the multipoint control device The target port is determined according to the push streaming parameters, and the target port can be adapted to the characteristics of the media content; further, the multipoint control device receives the push streaming data sent by the data access device through the target port, and pushes the push streaming data to the live broadcast device to Enable the live broadcast device to play the media content data in the push stream data during the web live broadcast process. In this way, the process of fusing media content from multiple data sources and performing webcasting is realized.
- the above method selects specific ports for media content from multiple data sources during the streaming process, it can make media content from multiple data sources It is well integrated and adapted to live broadcast equipment, reducing live broadcast delays caused by network fluctuations, and avoiding live streaming blockage caused by accidental conditions, and improving the stability of the web live broadcast process.
- FIG. 6 is a schematic structural diagram of a network device provided by an embodiment of this application.
- the network device 600 specifically includes:
- the access module 601 is configured to receive push data sent by at least two data sources;
- the streaming control module 602 is configured to determine streaming parameters according to the media content characteristics of the streaming data
- the streaming control module 602 is further configured to send the streaming parameter to the multipoint control device, and the streaming parameter is used to instruct the multipoint control device to determine a target port adapted to the media content feature ;
- the flow control module 602 is further configured to receive target port information sent by the multipoint control device;
- the access module 601 is further configured to send the push data through the target port.
- the flow control module 602 is further configured to send a mixed flow request to the multipoint control device according to the push streaming data, and the mixed flow request is used to indicate the The multipoint control device determines the set of free ports;
- the flow control module 602 is further configured to receive a set of free ports sent by the multipoint control device;
- the access module 601 is further configured to perform port synchronization according to the set of free ports.
- the access module 601 is also used to obtain the user type of the data source
- the access module 601 is further configured to determine the data source for accessing the target port according to the user type.
- the access module 601 is further configured to obtain page layout parameters sent by the data source;
- the access module 601 is further configured to send the page layout parameters to the multipoint control device, so that the multipoint control device adjusts the push data according to the page layout parameters.
- the above embodiment introduces the functional composition of the network device and the control server.
- the interaction of the functional modules in the network device and the control server will be described below in conjunction with the specific webcast system framework.
- Figure 7, which is an embodiment of the application. Provides another network live broadcast system framework diagram. Compared with the network live broadcast system framework shown in Figure 2, the multipoint control device in Figure 2 is further divided into a scheduling module, an adaptation module, an audio module, and a video module. And push streaming module.
- the scheduling module regularly collects load information, geographic location information, and machine life and death status of the adaptation module, audio module, video module, and push module, and then guides them based on the collected information.
- the load distribution of these modules, as well as the dynamic switching of machines in the event of a crash. Receive the request for mixing or pushing from the flow control module, and according to the geographic location information in the request, load balance and distribute the corresponding execution devices of the adaptation module, audio module, video module, and push module in the nearest location, and After requesting the execution device to allocate a port, the result is returned to the flow control module.
- the adaptation module it is specifically used to periodically report load information to the scheduling module. After receiving the allocation request from the dispatch module, it allocates ports and establishes a session locally, then receives the data sent from the interface machine, and distributes it to the back-end audio module and video module after processing.
- the audio module it is specifically used to select multiple channels of audio, perform audio mixing according to the needs of the live broadcaster, and transcode into the audio format required by the live broadcaster.
- the video module it is specifically used for routing and typesetting according to the needs of the live broadcaster, and transcoding into the corresponding video format, resolution, frame rate, and bit rate according to the needs of the live broadcaster.
- the streaming module it is specifically used to encapsulate the audio and video data processed by the audio module and the video module into an rtmp stream and push it to a live broadcast system.
- FIG. 8 it is a schematic diagram of an interaction of load balancing provided by an embodiment of this application.
- the adaptation module, audio module, video module, and streaming module periodically report load information to the scheduling module, for example: upload load information every second, where the load information includes: CPU usage rate, CPU core number , Inflow, Outflow, Inflow, Outflow; Then the dispatch module receives the load information of the four modules and the geographic location information of each machine configured, so as to grasp the load situation of each module in the world in real time.
- adaptation module can also mutually detect the disconnection of upstream and downstream modules; if disconnection is detected, it will be reported to the dispatching module, and the dispatching module will guide the machine according to the life and load conditions of the machine. Switch to the appropriate device.
- FIG. 9 is a schematic diagram of interaction of multi-terminal media content mixing provided in an embodiment of this application. Including the following steps:
- the flow control machine sends a mixed flow instruction to the scheduling module.
- the scheduling module allocates ports, that is, the scheduling module allocates the data adaptation device according to the global load information and the location information of the request source, and sends a port allocation request to the data adaptation device, and the data adaptation device establishes the user's session locally , And allocate free ports, and the result is returned to the scheduling module.
- the scheduling module returns the allocated port information of the adaptation module and the IP information of the device to the flow control machine.
- the flow control machine synchronizes the port information sent by the adaptation module and the IP information of the device to the interface machine.
- the interface machine After obtaining the port information of the adaptation module and the IP information of the device, the interface machine establishes a connection with the adaptation module, and sends data.
- FIG. 10 is a schematic diagram of the interaction of multi-terminal media content streaming provided by an embodiment of this application. Including the following steps:
- the stream control machine sends the push stream instruction to the scheduling module, and writes the corresponding push stream parameters, such as the audio audio format, number of channels, audio bit rate, etc., or the video encoding format, allocation rate, bit rate, and multiple Layout and other parameters of the road screen.
- the scheduling module allocates data adaptation, audio processing, video processing, and streaming machines based on the global load information and the location information of the request source, and sends to these machines allocating port requests, data adaptation, audio processing, video processing,
- the streaming machine establishes the user's session locally and allocates free ports, and the result is returned to the scheduling module.
- the uplink muxing session will be reused here, and if the user needs both muxing and push streaming, the two requests can be combined into one request.
- the audio processing and video modules save relevant audio and video parameters while establishing a local session.
- the scheduling module module returns the allocated data adaptation, the port information of the module, and the IP information of the device to the flow control machine.
- the flow control machine synchronizes the port information of the adaptation module and the IP information of the device to the interface machine.
- the figure shows two interface machines.
- the number of interface machines can also be determined according to the user type corresponding to the terminal device.
- the specific setting rules depend on the actual scenario and are not limited here.
- the interface machine After obtaining the port information of the adaptation module and the IP information of the device, the interface machine establishes a connection with the adaptation module, and sends data.
- the scheduling module synchronizes the allocated audio processing and video processing port information and the device's IP information to the adaptation module, and synchronizes the push streaming module port information and the device's IP information to the audio processing and video module.
- the adaptation module After receiving the audio processing and video processing port information and the device's IP information, the adaptation module establishes a connection with the audio processing and video processing device, and sends audio and video data.
- the streaming module establishes a connection with the live broadcast device, and after receiving the data from the audio module and the video module, it pushes the streaming rtmp data to the live broadcast system.
- the adaptation module can support the standard rtp protocol, and can also support various private audio and video transmission protocols.
- the streaming format pushed by the server MCU is not limited to rtmp and HLS.
- FIG. 11 shows another type of multiple data provided by this embodiment of the application.
- FIG. 11 shows another type of multiple data provided by this embodiment of the application.
- FIG. 11 Schematic diagram of the interaction of terminal media content push streaming.
- a 3-person real-time audio and video call room is used as an example to illustrate the uplink data processing process.
- the corresponding interface machine has three user ports A, B, and C. Among them, A and B have uplink data, that is, the host; C is the audience user , There is no need to push data. That is, the process includes:
- the interface machine obtains the type of the user.
- the interface machine allocates ports according to user types to connect with the adapter module.
- the adaptation module After the adaptation module receives the upstream audio and video data sent by A and B, it sorts the data packets and recovers the packet loss, and then transmits the decoded audio frames to the audio module frame by frame, and includes audio parameters such as audio format, The number of channels, etc., the decoded video frames are passed to the video module frame by frame, and video parameters such as encoding format and distribution rate are attached.
- the adaptation module needs to receive and process data of different protocols, such as standard rtp data, or private udp audio and video data.
- the audio module receives the audio data, it performs mixing and transcoding according to the saved streaming parameters, and then sends it to the streaming module.
- the video module After the video module receives the video data, it makes the screen layout according to the saved push streaming parameters, and performs mixing processing according to the resolution, bit rate, and frame rate required for the live broadcast, and then sends it to the push streaming module.
- the streaming module After receiving the audio and video data, the streaming module encapsulates the audio and video data into an rtmp format, and pushes the streaming to the live broadcast system.
- the method for managing webcast data can simultaneously push products of real-time multi-person audio and video calls to the live broadcast system and use the live broadcast service.
- a real-time multi-person call client can also be used to collect and push the live broadcast, so as to obtain a better push delay and weak network push effect.
- Push streaming can flexibly customize the resolution, bit rate, frame rate, and screen layout.
- the method and device in the above embodiment are specifically applied in a webcast scene where media content of multiple terminals is integrated.
- users can flexibly join the interactive process of media content of multiple terminals, such as the process of a multi-person video call, and then Push the video call content to the webcast system to facilitate the update of the media content during the webcast.
- FIG. 12 this is a flowchart of another method for managing webcast data provided by an embodiment of this application.
- the media content of the second terminal is being streamed to the third terminal for playback.
- the first terminal and the second terminal can generate webcast media content through the client SDK, and the third terminal can watch the webcast through the H5 page.
- the above process includes the following steps:
- the second terminal and the third terminal establish a first connection of media content.
- the first connection established between the second terminal and the third terminal means that the third terminal is playing the media content pushed by the second terminal.
- the first terminal sends a join request to the second terminal.
- the join request sent by the first terminal to the second terminal may include the streaming parameters of the first terminal, that is, the characteristic parameters of the media content generated by the first terminal, such as: screen resolution, bit rate, frame rate, Screen layout, etc.
- the second terminal applies to the control server for mixed streaming.
- the second terminal determines the target port for adaptation.
- the second terminal configures with the first terminal and synchronizes the target port to establish a second connection.
- the second terminal sends push streaming data to the third terminal.
- a third connection for media content is established between the first terminal, the second terminal, and the third terminal.
- step 1203 to step 1207 is similar to step 404 to step 413 of the embodiment shown in FIG.
- the third terminal plays the media content according to the push streaming data.
- FIG. 13 is a schematic diagram of updating a webcast page provided by an embodiment of this application.
- the live broadcast interface when the terminal establishes the first connection. After the media content of the first terminal in the above embodiment is added, the live broadcast interface can be re-arranged because the page layout parameters are included in the push streaming data, which is shown in Figure 13.
- the live interface on the right side realizes the integration process of multi-terminal media content and pushes it to the live broadcast system for playback, which improves the richness of the media content on the live page and has good stability when adding media content.
- the embodiment of the present application also provides a terminal device.
- FIG. 14 it is a schematic structural diagram of another terminal device provided by the embodiment of the present application.
- the terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc. Take the terminal as a mobile phone as an example:
- FIG. 14 shows a block diagram of a part of the structure of a mobile phone related to a terminal provided in an embodiment of the present application.
- the mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, and a processor 1480 , And power supply 1490 and other components.
- RF radio frequency
- the RF circuit 1410 can be used for receiving and sending signals during the process of sending and receiving information or talking. In particular, after receiving the downlink information of the base station, it is processed by the processor 1480; in addition, the designed uplink data is sent to the base station.
- the RF circuit 1410 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like.
- the RF circuit 1410 can also communicate with the network and other devices through wireless communication.
- the above-mentioned wireless communication can use any communication standard or protocol, including but not limited to global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (code division multiple access) multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.
- GSM global system of mobile communication
- GPRS general packet radio service
- code division multiple access code division multiple access
- CDMA code division multiple access
- WCDMA wideband code division multiple access
- LTE long term evolution
- email short messaging service
- the memory 1420 may be used to store software programs and modules.
- the processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420.
- the memory 1420 may mainly include a storage program area and a storage data area.
- the storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data created by the use of mobile phones (such as audio data, phone book, etc.), etc.
- the memory 1420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
- the input unit 1430 can be used to receive inputted number or character information, and generate key signal input related to the user settings and function control of the mobile phone.
- the input unit 1430 may include a touch panel 1431 and other input devices 1432.
- the touch panel 1431 also called a touch screen, can collect user touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on the touch panel 1431 or near the touch panel 1431. Operation, and air touch operation within a certain range on the touch panel 1431), and drive the corresponding connection device according to the preset program.
- the touch panel 1431 may include two parts: a touch detection device and a touch controller.
- the touch detection device detects the user's touch position, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 1480, and can receive and execute the commands sent by the processor 1480.
- the touch panel 1431 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave.
- the input unit 1430 may also include other input devices 1432.
- the other input device 1432 may include, but is not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackball, mouse, and joystick.
- the display unit 1440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone.
- the display unit 1440 may include a display panel 1441.
- the display panel 1441 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
- the touch panel 1431 can cover the display panel 1441. When the touch panel 1431 detects a touch operation on or near it, it is transmitted to the processor 1480 to determine the type of the touch event, and then the processor 1480 responds to the touch event. The type provides corresponding visual output on the display panel 1441.
- the touch panel 1431 and the display panel 1441 are used as two independent components to realize the input and input functions of the mobile phone, but in some embodiments, the touch panel 1431 and the display panel 1441 can be integrated. Realize the input and output functions of the mobile phone.
- the mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.
- the light sensor can include an ambient light sensor and a proximity sensor.
- the ambient light sensor can adjust the brightness of the display panel 1441 according to the brightness of the ambient light.
- the proximity sensor can close the display panel 1441 and/or when the mobile phone is moved to the ear. Or backlight.
- the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary.
- the audio circuit 1460, the speaker 1461, and the microphone 1462 can provide an audio interface between the user and the mobile phone.
- the audio circuit 1460 can transmit the electric signal after the conversion of the received audio data to the speaker 1461, which is converted into a sound signal by the speaker 1461 for output; on the other hand, the microphone 1462 converts the collected sound signal into an electric signal, and the audio circuit 1460 After being received, it is converted into audio data, and then processed by the audio data output processor 1480, and then sent to, for example, another mobile phone via the RF circuit 1410, or the audio data is output to the memory 1420 for further processing.
- WiFi is a short-distance wireless transmission technology.
- the mobile phone can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 1470. It provides users with wireless broadband Internet access.
- FIG. 14 shows the WiFi module 1470, it is understandable that it is not a necessary component of the mobile phone, and can be omitted as needed without changing the essence of the invention.
- the processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. Various functions and processing data of the mobile phone can be used to monitor the mobile phone as a whole.
- the processor 1480 may include one or more processing units; optionally, the processor 1480 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application programs. And so on, the modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 1480.
- the mobile phone also includes a power supply 1490 (such as a battery) for supplying power to various components.
- a power supply 1490 (such as a battery) for supplying power to various components.
- the power supply can be logically connected to the processor 1480 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.
- the mobile phone may also include a camera, a Bluetooth module, etc., which will not be repeated here.
- the processor 1480 included in the terminal also has the function of executing each step of the above-mentioned method for managing webcast data.
- FIG. 15 is a schematic structural diagram of another control server provided by an embodiment of the present application. Differences may include one or more central processing units (CPU) 1522 (for example, one or more processors) and memory 1532, and one or more storage media 1530 (for example, application programs 1542 or data 1544) One or one storage device in Shanghai). Among them, the memory 1532 and the storage medium 1530 may be short-term storage or permanent storage.
- the program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of command operations on the server.
- the central processing unit 1522 may be configured to communicate with the storage medium 1530, and execute a series of instruction operations in the storage medium 1530 on the server 1500.
- the server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and/or one or more operating systems 1541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
- operating systems 1541 such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
- the steps performed by the apparatus for managing webcast data in the foregoing embodiment may be based on the server structure shown in FIG. 15.
- An embodiment of the present application also provides a computer-readable storage medium, which stores data management instructions, which when run on a computer, causes the computer to execute the embodiments shown in FIGS. 3 to 4 above. The steps performed by the multipoint control device or data access device in the described method.
- the embodiment of the present application also provides a computer program product including data management instructions, which when running on a computer, causes the computer to execute the multi-point control device or data in the method described in the foregoing embodiments shown in FIGS. 3 to 4 Steps taken to access the device.
- the embodiment of the present application also provides a data management system, and the data management system may include the control server in the embodiment described in FIG. 5 or the network device described in FIG. 6.
- the disclosed system, device, and method can be implemented in other ways.
- the device embodiments described above are only illustrative.
- the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
- the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
- the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
- the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
- the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
- the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a data management device, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
- the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Human Computer Interaction (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Transfer Between Computers (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
本申请公开了一种网络直播数据的管理方法以及相关装置,通过接收数据接入设备发送的推流请求,并确定数据接入设备中来自多个数据来源的推流数据的媒体内容特征设定的推流参数;然后根据推流参数确定目标端口;进而通过目标端口接收数据接入设备发送的推流数据,并向直播设备推送推流数据,以使直播设备在进行网络直播的过程中播放媒体内容数据。从而实现了融合多终端媒体内容并进行网络直播的过程,减少了网络波动造成的直播延迟,且避免了由于偶发状况造成的直播推流阻塞,提高了网络直播过程的稳定性。
Description
本申请要求于2020年02月20日提交中国专利局、申请号为2020101043781、申请名称为“一种网络直播数据的管理方法以及相关装置”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及互联网技术领域,尤其涉及网络直播数据的管理技术。
随着互联网技术的发展,网络直播间以新颖的形式、丰富的内容吸引了越来越多用户。通常不同的网络直播间用于播放不同的直播内容,终端通过接入不同的网络直播间,相应地满足用户不同的观看需求。
为了丰富网络直播的内容,可以将终端与其他终端的交互过程进行视频推流,例如:对当前多人音视频通话的内容进行直播推流,经过转码或多路流混流后,分发到内容分发网络(Content Delivery Network,CDN),然后其他用户即可通过信息传输协议拉流观看。
但是,由于涉及对于多个终端的音视频推流,如若其中某一音视频源的网络发生波动,则可能对整个直播过程产生影响,容易造成直播延迟;并且,当某一音视频源发生故障时,甚至可能造成直播推流阻塞,影响多终端融合的网络直播过程的稳定性。
发明内容
有鉴于此,本申请提供一种网络直播数据的管理方法及相关装置,可以有效避免因多个媒体内容之间的参数差异或网络环境差异而造成的网络直播延迟,提高网络直播过程的稳定性。
本申请第一方面提供一种网络直播数据的管理方法,由多点控制设备执行,所述方法具体包括:
接收数据接入设备发送的推流请求,所述推流请求包括推流参数,所述推流参数是所述数据接入设备基于待发送的推流数据的媒体内容特征确定的,所述推流数据包括来自至少二个数据来源的媒体内容数据;
根据所述推流参数确定适配于所述媒体内容特征的目标端口;
通过所述目标端口接收所述数据接入设备发送的所述推流数据;
向直播设备推送所述推流数据,以使所述直播设备在进行网络直播过程中播放所述媒体内容数据。
本申请第二方面提供另一种数据管理的方法,由数据接入设备执行,所述方法具体包括:
接收至少二个数据来源发送的推流数据;
根据所述推流数据的媒体内容特征确定推流参数;
向多点控制设备发送所述推流参数,所述推流参数用于指示所述多点控制设备确定适配于所述媒体内容特征的目标端口;
接收所述多点控制设备发送的所述目标端口的信息;
通过所述目标端口发送所述推流数据。
本申请第三方面提供一种控制服务器,所述控制服务器具体包括:
调度模块,用于接收数据接入设备发送的推流请求,所述推流请求包括推流参数,所述推流参数是所述数据接入设备基于待发送的推流数据的媒体内容特征确定的,所述推流数据包括来自至少二个数据来源的媒体内容数据;
适配模块,用于根据所述推流参数,确定适配于所述媒体内容特征的目标端口;
媒体内容模块,用于通过所述目标端口接收所述数据接入设备发送的所述推流数据;
推流模块,用于向直播设备推送所述推流数据,以使所述直播设备在进行网络直播过程中播放所述媒体内容数据。
本申请第四方面提供一种网络设备,可以应用于网络直播***中,所述网络设备具体包括:
接入模块,用于接收至少二个数据来源发送的推流数据;
流控模块,用于根据所述推流数据的媒体内容特征确定对应的推流参数;
所述流控模块,还用于向所述多点控制设备发送所述推流参数,所述推流参数用于指示所述多点控制设备确定适配于所述媒体内容特征的目标端口;
所述流控模块,还用于接收所述多点控制设备发送的目标端口的信息;
所述接入模块,还用于通过所述目标端口发送所述推流数据。
本申请第五方面提供一种计算机设备,包括:存储器、处理器以及总线***;所述存储器用于存储程序代码;所述处理器用于根据所述程序代码中的指令执行上述第一方面所述的网络直播数据的管理方法,或上述第二方面所述的网络直播数据的管理方法。
本申请第六方面提供一种计算机可读存储介质,所述计算机可读存储介质中存储有指令,当其在计算机上运行时,使得计算机执行上述第一方面所述的网络直播数据的管理方法,或上述第二方面所述的网络直播数据的管理方法。
本申请第七方面提供一种计算机程序产品,包括指令,当其在计算机上运行时,使得计算机执行述第一方面所述的网络直播数据的管理方法,或上述第二方面所述的网络直播数据的管理方法。
通过多点控制设备接收数据接入设备发送的推流请求,并确定基于数据接入设备中来自多个数据来源的推流数据的媒体内容特征设定的推流参数;然后,多点控制设备根据推流参数确定目标端口,该目标端口能够适配于推 流数据的媒体内容特征;进而,多点控制设备通过目标端口接收数据接入设备发送的推流数据,并向直播设备推送该推流数据,以使直播设备在进行网络直播过程中播放推流数据中的媒体内容数据。。
图1为数据管理***运行的网络架构图;
图2为本申请实施例提供的一种网络直播的***架构图;
图3为本申请实施例提供的一种网络直播数据的管理方法的流程图;
图4为本申请实施例提供的另一种网络直播数据的管理方法的流程图;
图5为本申请实施例提供的一种控制服务器的结构示意图;
图6为本申请实施例提供的一种网络设备的结构示意图;
图7为本申请实施例提供的另一种网络直播的***框架图;
图8为本申请实施例提供的一种负载均衡的交互示意图;
图9为本申请实施例提供的一种多终端媒体内容混流的交互示意图;
图10为本申请实施例提供的一种多终端媒体内容推流的交互示意图;
图11为本申请实施例提供的另一种多终端媒体内容推流的交互示意图;
图12为本申请实施例提供的另一种网络直播数据的管理方法的流程图;
图13为本申请实施例提供的一种网络直播页面的更新示意图;
图14为本申请实施例提供的另一种终端设备的结构示意图;
图15为本申请实施例提供的另一种控制服务器的结构示意图。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施例例如能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“对应于”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、***、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
首先,对本申请实施例中可能出现的一些名词进行解释。
多点控制单元(Multipoint Control Unit,MCU):在本申请中又被称为多点控制设备,其是音视频会议***的核心部分,主要功能是为用户提供群组会议、多组会议的连接服务。
选择转发单元(Selective Forwarding Unit,SFU):单纯地用于对音视频数据进行转发,不涉及音视频数据格式、参数等转换及合并,对于音视频数据格式、参数等的转换及合并由观看端收到数据后处理。
软件开发工具包(Software Development Kit,SDK):是辅助开发某一 类软件的相关文档、范例和工具的集合。
混流:是对于视频和音频数据流的控制、同步以及混合的方式,即将视频和音频数据合并。
推流:是把采集阶段封装好的内容数据传输到指定设备的过程。
应理解,本申请提供的数据管理方法可以应用于支持网络直播功能的***或程序中,例如直播平台,具体的,该支持网络直播功能的***可以运行于如图1所示的网络架构中,如图1所示,是***运行的网络架构图,由1图可知,该***可以与多个终端设备进行媒体内容的交互,终端通过网络建立与服务器的连接,服务器对终端上传的音视频流中转推流至目标终端,以实现网络直播;可以理解的是,图1中示出了多种终端设备,在实际场景中可以有更多或更少种类的终端设备参与到网络直播的过程中,具体数量和种类因实际场景而定,此处不做限定,另外,图1中示出了一个服务器,但在实际场景中,也可以有多台服务器的参与,特别是在多内容应用交互的场景中,具体服务器的数量因实际场景而定。
可以理解的是,上述网络直播数据的管理***可以是以一种程序的形式在服务器中运行,也可以作为服务器中的***部件进行运行,还可以作为云端服务程序的一种,具体运作模式因实际场景而定,此处不做限定。
针对相关技术中多终端媒体内容融合的网络直播过程不稳定的问题,本申请实施例提供了一种网络直播数据的管理方法,该方法针对多终端媒体内容融合的网络直播,在推流过程中对多终端的媒体内容进行了针对性的端口选择,由此使得多终端的媒体内容更好地融合,并且适配于直播设备,减少了网络波动造成的直播延迟,且避免了因偶发状况造成的直播推流阻塞,提高了网络直播过程的稳定性。
本申请实施例提供的网络直播数据的管理方法可以应用于图2所示的数据管理的流程框架中,如图2所示,为本申请实施例提供的一种网络直播的***架构图,下面对图2中出现的相关设备进行说明:
接口机:用于中转终端设备发送的音视频数据及音视频过程中的信令。
流控机:用于实时调节音视频过程中的各项音视频参数和音频选路。
配置中心:包含所有机器的位置分布以及实时负载情况,并且可以将所有机器的位置分布以及实时负载情况同步到信息中心。
策略调度服务器:负责请求鉴权、签名加密、安全确认等业务。根据用户位置信息以及机器的位置、负载信息就近分配接口机、流控机,使得同房间同运营商的用户尽量聚集到相同的接口机。
查询服务:包括但不限于查询用户资料、关系链、备注、分组、群关系链、用户产生的信息等。
登录服务:为快速完成帐号集成接入音视频或即时通信云服务而提供的通用帐号登录组件。
信令服务器:主要负责web侧用户的接入和登录、信令的透传、中转服务器的分配、音视频初始参数的协商下发、连接通道的协商及安全参数的协商下发。
中转服务器:首先,用于面向web用户模拟网页即时通讯与web建立直连,包括互动式连接的建立、数据包传输层安全性协议密钥的协商、安全实时传输协议音视频数据的中转及基于实时传输控制(Real-timeControlProtocol,RTCP)协议的音视频质量控制。其次,用于面向接口机模拟SDK用户与现有互动直播***建立连接,包括进入房间、数据传输等。再次,用于将web用户和SDK用户的进出房间、状态通知等信令衔接起来,进而对web用户和SDK用户双方的音视频数据转码、转封装进行衔接,以换取统一的标识ID,兼容互动直播的房间管理和账号***。
本申请实施例提供的网络直播数据的管理方法涉及了包括接口机、多点控制设备以及直播设备中的中转服务器之间的交互过程,即相当于在接口机与中转服务器之间添加了多点控制设备,以对网络直播过程中的媒体数据进行管理。
可以理解的是,本申请所提供的方法可以为一种程序的写入,以作为硬件***中的一种处理逻辑,也可以作为一种数据管理装置,采用集成或外接的方式实现上述处理逻辑。作为一种实现方式,该网络直播数据的管理装置通过多点控制设备接收数据接入设备发送的推流请求,并确定基于数据接入设备中来自多个数据来源的推流数据的媒体内容特征设定的推流参数;然后,多点控制设备根据推流参数确定目标端口,该目标端口能够适配于推流数据的媒体内容特征;进而,多点控制设备通过该目标端口接收数据接入设备发送的推流数据,并向直播设备推送该推流数据,以使直播设备在进行网络直播过程中播放推流数据的媒体内容。如此,实现了融合多终端媒体内容并且进行网络直播的过程,由于上述方法在推流过程中对多数据来源的媒体内容进行了针对性的端口选择,因此,可以使得多数据来源的媒体内容很好的融合,并且适配于直播设备,减少了网络波动造成的直播延迟,且避免了由于偶发状况造成的直播推流阻塞,提高了网络直播过程的稳定性。
结合上述流程架构,下面将对本申请中网络直播数据的管理方法进行介绍。请参阅图3,图3为本申请实施例提供的一种网络直播数据的管理方法的流程图,该方法应用于网络直播***中,网络直播***包括数据接入设备、多点控制设备和直播设备,本申请实施例至少包括以下步骤:
301、数据接入设备根据推流数据的媒体内容特征确定推流参数。
本实施例中,推流数据包括来自至少二个数据来源的媒体内容数据,例如:推流数据可以为三人视频通话的媒体内容数据;而推流参数基于媒体内容数据的特征确定的参数,例如:音频的格式、声道数、音频码率等,或视 频的编码格式、分配率、码率、多路画面的排版等参数。
另外,数据来源可以是用于实时上传音频、视频等媒体内容数据的终端设备,例如实时上传所录制的视频的手机等;数据来源也可以是网络中提供数据资源的平台,例如提供视频内容的多个不同的网站,具体形式因实际场景而定。
可以理解的是,具体的推流参数可以是上述实例中的一种或多种参数的组合,可以根据推流数据所处的应用场景设定,例如:若推流数据为多人音频通话的音频内容数据,则推流参数应当为音频的格式、声道数、音频码率;具体的参数形式因实际场景而定,此处不做限定。
通过获取推流参数,可以更好地了解来自不同数据来源的媒体内容之间的参数差异,进而可以有针对性地进行端口选择,从而保证更好地将网络直播过程中来自多数据来源的媒体内容结合起来。
本实施例中,确定推流参数的过程可以基于人工智能模型实现,即一旦推流数据接入,则可通过预设的分析模型对推流数据中的各项参数进行提取,例如,假设网络直播的需求为高音质,则通过分析模型分析出的推流参数应当包括音频的格式、声道数、音频码率,并提取具体数据。
302、数据接入设备向多点控制设备发送推流请求。
本实施例中,推流请求中包含了推流参数。
303、多点控制设备接收推流请求并获取推流参数。
304、多点控制设备根据推流参数,确定适配于媒体内容特征的目标端口。
本实施例中,多点控制设备可以根据全局的负载信息和请求来源的位置信息分配数据适配设备、音频处理设备、视频处理设备和推流设备,并向这些设备发送分配端口请求,数据适配设备、音频处理设备、视频处理设备和推流设备在本地建立会话,并分配目标端口。
可选的,多点控制设备关联了网络直播***中的多个数据适配设备、音频处理设备、视频处理设备和推流设备,在确定目标端口之前,多点控制设备会收集上述关联设备的负载情况,具体可以包括CPU使用率、CPU核数、入流量、出流量、入流数、出流数等,然后根据预设条件选择对应的设备,例如:推流参数中指示所需的关联设备的CPU使用率小于50%,则多点控制设备可以在关联设备中遍历查找CPU使用率小于50%的设备,并确定该设备的地址以及对应的端口。
可以理解的是,上述关联设备也可以是集成于多点控制设备中的设备,具体的设备形式因实际场景而定,此处不做限定。
通过上述方式确定目标端口,可以保证网络直播过程中的网络流畅度,即根据不同终端所需的媒体内容参数选择对应的目标端口,基于该目标端口实现多终端的媒体内容融合及推流的过程,可以保证网络直播过程中的网络流畅度。
305、多点控制设备向数据接入设备发送目标端口信息。
本实施例中,多点控制设备根据上述步骤304中确定的目标端口,确定对应的端口标识以及端口对应的设备地址信息,将这些信息打包为目标端口信息发送给数据接入设备。
306、多点控制设备与数据接入设备同步配置目标端口,以建立连接。
本实施例中,同步配置目标端口的过程即为根据设备地址信息确定对应设备,以及根据端口标识进行端口的选择的过程,确定设备以及端口之后即可建立连接。
307、数据接入设备向多点控制设备发送推流数据。
本实施例中,基于上述步骤306中建立的端口连接,数据接入设备向多点控制设备发送推流数据。
308、多点控制设备向直播设备发送推流数据。
本实施例中,在多点控制设备向直播设备发送推流数据之前,还可以确定推流数据的媒体内容类型,例如音频或视频;然后根据媒体内容类型对推流数据进行分类,确定推流数据的分类类型;进而,将推流数据分别转换成与该分类类型对应的目标格式,并推送至直播设备。例如:目标格式要求的音频数据格式为MP3,视频数据格式为FLV,则对推流数据进行分类为音频数据和视频数据后,分别转换至目标格式。
可以理解的是,媒体内容分类的过程可以是基于语音识别进行的,即可以自动识别出推流数据中音频的部分,并将其切换至相应的格式,从而保证了推流过程的针对性。
309、直播设备根据推流数据进行媒体内容的播放。
本实施例中,推流至直播设备的推流数据中融合了来自至少二个数据来源的媒体内容数据,其中,媒体内容数据之间可以是并列的,也可以是层叠的,还可以响应于用户对于与直播设备关联的播放页面触发的操作相应地进行媒体内容的切换,例如:直播设备与H5页面关联,则直播设备可以根据用户在H5页面对于媒体内容的选择情况播放对应的媒体内容。
结合上述实施例可知,通过多点控制设备接收数据接入设备发送的推流请求,并确定基于数据接入设备中来自多个数据来源的推流数据的媒体内容特征设定的推流参数;然后,多点控制设备根据推流参数确定目标端口,该目标端口能够适配于推流数据的媒体内容特征;进而,多点控制设备通过该目标端口接收数据接入设备发送的推流数据,并向直播设备推送该推流数据,以使直播设备在进行网络直播过程中播放推流数据的媒体内容。如此,实现了融合多数据来源的媒体内容并且进行网络直播的过程,由于上述方法在推流过程中对多数据来源的媒体内容进行了针对性的端口选择,因此,可以使得多数据来源的媒体内容很好的融合,并且适配于直播设备,减少了网络波动造成的直播延迟,且避免了由于偶发状况造成的直播推流阻塞,提高了网 络直播过程的稳定性。
上述实施例介绍了网络直播数据的管理的过程,下面,结合游戏应用作为具体场景进行介绍,请参阅图4,图4为本申请实施例提供的另一种网络直播数据的管理的方法的流程图,应用于网络直播***中,网络直播***包括数据接入设备、多点控制设备和直播设备,本申请实施例至少包括以下步骤:
401、多点控制设备周期性进行负载均衡。
本实施例中,多点控制设备进行负载均衡的过程即按照预设周期获取自身的关联设备的负载信息,该负载信息用于指示关联设备的异常情况;若负载信息不满足预设条件,则根据负载信息确定发生异常的关联设备;然后对发生异常的关联设备进行地址切换,以使得该关联设备的负载信息满足预设条件。例如:假设预设条件为多点控制设备与关联设备之间的响应延迟小于10毫秒,则应当检测响应延迟小于10毫秒的关联设备。
可以理解的是,关联设备可以是通过网络与多点控制设备关联,并能够实现相应功能的设备;关联设备也可以是集成在多点控制设备本地的多个设备,具体的关联形式因实际场景而定。
402、数据接入设备向多点控制设备发送混流请求。
本实施例中,由于一般网络直播中的媒体内容为视频与音频的结合内容,为保证视频与音频的对应关系,需要进行混流过程。
403、多点控制设备确定空闲端口集合。
本实施例中,在混流的过程中可以确定多点控制设备的空闲端口集合,即可用的关联设备以及可用的关联设备上的端口。
404、数据接入设备根据推流数据确定推流参数。
405、数据接入设备向多点控制设备发送推流请求。
本实施例中,发送推流请求的过程可以复用混流过程的会话,并且如果用户是混流和推流都需要的,两个请求可以合并为一个请求。
406、多点控制设备接收推流请求并获取推流参数。
407、多点控制设备根据推流参数确定适配的目标端口。
本实施例中,目标端口的选择可以基于混流过程中确定的空闲端口集合进行,如此减小了端口判断的范围,提高了目标端口确定的效率。
408、多点控制设备向数据接入设备发送目标端口信息。
409、多点控制设备与数据接入设备同步目标端口,以建立连接。
410、数据接入设备获取数据来源的用户类型,以确定对应的目标端口。
411、数据接入设备向多点控制设备发送推流数据。
本实施例中,步骤408-步骤411与图3所述实施例中的步骤305-步骤308相似,相关特征可以进行参考,此处不做赘述。
412、多点控制设备将推流数据调整为目标格式以及目标页面布局。
本实施例中,目标格式以及目标页面布局可以是直播设备发送的,也可以是在多点控制设备的历史数据处理记录中记录的。若目标格式以及目标页面布局是直播设备发送的,则多点控制设备需要接收直播设备发送的目标参数,该目标参数是基于网络直播过程中的媒体内容特征确定的,然后根据目标参数对推流数据进行调整。
可选的,目标格式以及目标页面布局也可以是与数据接入设备关联的终端设备设定的,具体的,多点控制设备可以接收数据接入设备发送的页面布局参数,该页面布局参数用于指示数据接入设备中来自至少二个数据来源的媒体内容数据的页面分布情况;然后根据页面布局参数对推流数据进行调整,例如:画中画、四宫格、九宫格等页面布局方式。从而灵活地对来自多个数据来源的媒体内容数据进行融合,提高了网络直播的丰富度。
可以理解的是,目标参数的作用对象可以是音频、视频或音频视频的结合;例如,在语音直播的场景中,目标参数可以为音频格式信息;在远程桌面等只有视频的场景中,目标参数可以为视频格式信息;在网络直播的场景中,目标参数可以包含音频格式信息和视频格式信息。通过对目标参数进行有针对性地设置,使得在不同的场景中都可以保证直播过程的正常进行。
另外,对于页面布局参数,其作用对象可是音频、视频或音频视频的结合;例如实现音频的混音、视频的“画中画”等效果,提高了直播过程的灵活性。
在另一种可能的场景中,对于目标参数或页面布局参数的作用对象也可以是基于人工智能技术分析得到的,例如,网络直播***可以自动分析推流数据的类型,当确定推流数据为音频数据时,可以自动提取目标参数中对应于音频的部分,并进行相关的设定。
413、多点控制设备向直播设备发送调整后的推流数据。
本实施例中,为保证推流过程的保密性,可以对推流数据进行加密,其中,加密的过程可以基于策略调度设备发送的目标密钥实现,该目标密钥用于指示直播设备的地址,为推流数据设置目标密钥,进而根据该目标密钥对推流数据进行加密。
414、直播设备根据推流数据进行媒体内容的播放。
结合上述实施例可见,在基于服务器选择转发单元(Selective Forwarding Unit,SFU)方案的实时音视频引擎(Real-Time Communication,RTC)的基础上,本申请提供的网络直播数据的管理方法可以通过多点控制设备来推送实时消息传输协议(Real Time Messaging Protocol,rtmp)流或HLS(HTTP Live Streaming)流到直播***,从而实现在实时多人通话过程中同时直播。进一步的,基于对相关端口的配置的优化以及关联设备的选择的优化,从而可以实现更低延迟的直播,经统计通过rtmp、hls等格式推流并观看可以实现1~3 秒延迟,通过H5页面的观看可以实现0.5~1秒的延迟。由于多点控制设备对于推流过程的适配性,在弱网场景可以获得更好的直播体验,且可以灵活地定制分辨率、码率、帧率、画面布局,提高直播内容的丰富度。
为了更好的实施本申请实施例的上述方案,下面对上述网络直播数据的管理方法的相关装置进行进一步的说明。请参阅图5,图5为本申请实施例提供的一种控制服务器的结构示意图,相当于上述实施例中的多点控制设备,控制服务器500包括:
调度模块501,用于接收数据接入设备发送的推流请求,所述推流请求包括推流参数,所述推流参数是所述数据接入设备基于待发送的推流数据的媒体内容特征确定的,所述推流数据包括来自至少二个数据来源的媒体内容数据;
适配模块502,用于根据所述推流参数,确定适配于所述媒体内容特征的目标端口;
媒体内容模块503,用于通过所述目标端口接收所述数据接入设备发送的所述推流数据;
推流模块504,用于向直播设备推送所述推流数据,以使所述直播设备在进行网络直播过程中播放所述媒体内容数据。
可选的,在本申请一些可能的实现方式中,所述调度模块501,还用于按照预设周期获取所述多点控制设备的关联设备的负载信息,所述负载信息用于指示所述关联设备的异常情况;
所述调度模块501,还用于若所述负载信息不满足预设条件,则根据所述负载信息确定发生异常的关联设备的地址信息,所述预设条件是基于所述关联设备与所述多点控制设备的连接情况以及所述关联设备的负载情况确定;
所述调度模块501,还用于根据所述地址信息进行所述关联设备的地址切换,以使所述关联设备的负载信息满足所述预设条件。
可选的,在本申请一些可能的实现方式中,所述适配模块502,具体用于接收所述数据接入设备发送的混流请求;
所述适配模块502,具体用于根据所述混流请求确定空闲端口集合;
所述适配模块502,具体用于根据所述推流参数在所述空闲端口集合中选取所述目标端口。
可选的,在本申请一些可能的实现方式中,所述推流模块504,具体用于确定所述推流数据的媒体内容类型;
所述推流模块504,具体用于根据所述媒体内容类型对所述推流数据进行分类,确定所述推流数据的分类类型;
所述推流模块504,具体用于将所述推流数据转换成与所述分类类型对应的目标格式,并推送至所述直播设备。
可选的,在本申请一些可能的实现方式中,所述调度模块501,还用于接收所述直播设备发送的目标参数,所述目标参数是基于所述网络直播过程中的媒体内容特征确定的;
所述媒体内容模块503,还用于根据所述目标参数对所述推流数据进行调整。
可选的,在本申请一些可能的实现方式中,所述调度模块501,还用于接收所述数据接入设备发送的页面布局参数,所述页面布局参数用于指示所述数据接入设备中来自所述至少二个数据来源的媒体内容数据的页面分布情况;
所述媒体内容模块503,还用于根据所述页面布局参数对所述推流数据进行调整。
可选的,在本申请一些可能的实现方式中,所述网络直播***还包括策略调度设备,所述调度模块501,还用于获取策略调度设备发送的目标密钥,所述目标密钥用于指示所述直播设备的地址;
所述推流模块504,还用于根据所述目标密钥对所述推流数据进行加密。
通过多点控制设备接收数据接入设备发送的推流请求,并确定基于数据接入设备中来自多个数据来源的推流数据的媒体内容特征设定的推流参数;然后,多点控制设备根据推流参数确定目标端口,该目标端口能够适配于媒体内容特征;进而,多点控制设备通过该目标端口接收数据接入设备发送的推流数据,并向直播设备推送推流数据,以使直播设备在进行网络直播过程中播放推流数据中的媒体内容数据。如此,实现了融合多数据来源媒体内容并且进行网络直播的过程,由于上述方法在推流过程中对多数据来源的媒体内容进行了针对性的端口选择,因此,可以使得多数据来源的媒体内容很好的融合,并且适配于直播设备,减少了网络波动造成的直播延迟,且避免了由于偶发状况造成的直播推流阻塞,提高了网络直播过程的稳定性。
本申请还提供一种网络设备,即上述实施例中的数据接入设备,可以应用于网络直播***中,请参阅图6,图6为本申请实施例提供的一种网络设备的结构示意图,所述网络设备600具体包括:
接入模块601,用于接收至少二个数据来源发送的推流数据;
流控模块602,用于根据所述推流数据的媒体内容特征确定推流参数;
所述流控模块602,还用于向所述多点控制设备发送所述推流参数,所述推流参数用于指示所述多点控制设备确定适配于所述媒体内容特征的目标端口;
所述流控模块602,还用于接收所述多点控制设备发送的目标端口的信息;
所述接入模块601,还用于通过所述目标端口发送所述推流数据。
可选的,在本申请一些可能的实现方式中,所述流控模块602,还用于根据所述推流数据向所述多点控制设备发送混流请求,所述混流请求用于指示所述多点控制设备确定空闲端口集合;
所述流控模块602,还用于接收所述多点控制设备发送的空闲端口集合;
所述接入模块601,还用于根据所述空闲端口集合进行端口同步。
可选的,在本申请一些可能的实现方式中,所述接入模块601,还用于获取所述数据来源的用户类型;
所述接入模块601,还用于根据所述用户类型确定接入所述目标端口的数据来源。
可选的,在本申请一些可能的实现方式中,所述接入模块601,还用于获取所述数据来源发送的页面布局参数;
所述接入模块601,还用于向所述多点控制设备发送所述页面布局参数,以使所述多点控制设备根据所述页面布局参数对所述推流数据进行调整。
上述实施例介绍了网络设备和控制服务器的功能组成,下面结合具体的网络直播***框架对网络设备和控制服务器中各功能模块的交互情况进行说明,请参阅图7,图7为本申请实施例提供的另一种网络直播的***框架图,与图2所示的网络直播的***框架相比,将图2中的多点控制设备进一步划分为了调度模块、适配模块、音频模块、视频模块以及推流模块。
具体的,在图7所示的***框架中,调度模块定期收集适配模块、音频模块、视频模块以及推流模块的负载信息、地理位置信息、机器死活状态,进而,根据所收集的信息指导这些模块的负载分配,以及在发生死机时动态切换机器。接收流控模块发来的用于申请混流或推流的请求,依据请求中的地理位置信息,就近负载均衡分配适配模块、音频模块、视频模块以及推流模块各自对应的执行设备,并在向执行设备请求分配端口后,返回结果到流控模块。
对于适配模块,具体用于定期上报负载信息给调度模块。从调度模块收到分配请求后,分配端口并在本地建立会话,之后接收从接口机发送过来的数据,处理后分发到后端的音频模块、视频模块。
对于音频模块,具体用于对多路音频进行选路,依据直播方的需要做音频混流,以及转码成直播方需要的音频格式。
对于视频模块,具体用于依据直播方需要做选路及排版,并按直播方需求转码为相应的视频格式、分辨率、帧率、码率。
对于推流模块,具体用于将音频模块和视频模块处理后的音视频数据,封装成rtmp流,并推送的直播***。
下面结合图7所示的网络直播的***框架,对图3或图4中涉及的网络 直播数据管理方法对应的模块的交互过程进行说明。如图8所示,为本申请实施例提供的一种负载均衡的交互示意图。具体的,适配模块、音频模块、视频模块、推流模块周期性地向调度模块上报负载信息,例如:每秒都进行负载信息的上传,其中,负载信息包括:CPU使用率、CPU核数、入流量、出流量、入流数、出流数;然后调度模块接收四个模块的负载信息以及配置的各机器的地理位置信息,从而实时掌握全局各模块各地域的负载情况。
进一步的,适配模块、音频模块、视频模块、推流模块还可以相互检测上下游模块的连接断开情况;若检测到断开,则上报到调度模块,调度模块依据机器死活及负载情况指导切换合适的设备。
另外,对于推流数据的混流过程如图9所示,为本申请实施例提供的一种多终端媒体内容混流的交互示意图。包括如下步骤:
901、流控机发送混流指令到调度模块。
902、调度模块分配端口,即调度模块依据全局的负载信息和请求来源的位置信息分配数据适配设备,并向该数据适配设备发送分配端口请求,数据适配设备在本地建立该用户的会话,并分配空闲端口,结果返回给调度模块。
903、调度模块将分配的适配模块端口信息以及设备的IP信息返回给流控机。
904、流控机将适配模块发送的端口信息以及设备的IP信息同步到接口机。
905、接口机得到适配模块的端口信息以及设备的IP信息后建立与适配模块的连接,并发送数据。
另外,在推流过程中,具体的模块交互过程如图10所示,为本申请实施例提供的一种多终端媒体内容推流的交互示意图。包括如下步骤:
1001、流控机发送推流指令到调度模块,并写入相应的推流参数,如音频的音频格式、声道数、音频码率等,或视频的编码格式、分配率、码率、多路画面的排版等参数。
1002、调度模块依据全局的负载信息和请求来源的位置信息分配数据适配、音频处理、视频处理、推流的机器,并向这些机器发送分配端口请求,数据适配、音频处理、视频处理、推流的机器在本地建立该用户的会话,并分配空闲端口,结果返回给调度模块。
可以理解的是,如果该用户在适配模块已经存在上行混流的会话,这里会复用上行混流的会话,并且如果用户是混流和推流都需要的,两个请求可以合并为一个请求。音频处理和视频模块在建立本地会话的同时,保存相关音视频参数。
1003、调度模块模块将分配的数据适配、模块的端口信息以及设备的IP信息返回给流控机。
1004、流控机将适配模块的端口信息以及设备的IP信息同步到接口机。
本实施例中,图中示出了两个接口机,在实际场景中可以有更多或更少的接口机参与,具体的可以参考与网络设备关联的终端设备的数量设定接口机的数量;也可以根据终端设备对应的用户类型确定接口机的数量,具体的设定规则因实际场景而定,此处不做限定。
1005、接口机得到适配模块的端口信息以及设备的IP信息后建立与适配模块的连接,并发送数据。
1006、调度模块将分配的音频处理和视频处理的端口信息以及设备的IP信息同步到适配模块,将推流模块端口信息以及设备的IP信息同步到音频处理和视频模块。
1007、适配模块拿到音频处理和视频处理的端口信息以及设备的IP信息后,与音频处理和视频处理的设备建立连接,并发送音视频数据。
1008、推流模块与直播设备建立连接,收到了音频模块和视频模块的数据后,推流rtmp数据到直播***。
可以理解的是,适配模块可支持标准的rtp协议,也可以支持各类私有音视频传输协议。服务器MCU推流出的流格式不局限于rtmp、HLS。
可选的,考虑到与网络设备关联的终端设备可能是主播也可能是观众,主播需要进行数据推流,而观众不需要,如图11所示,为本申请实施例提供的另一种多终端媒体内容推流的交互示意图。图中以一个3人实时音视频通话房间为例说明上行数据处理过程,对应的接口机上有A、B、C三个用户端口,其中A、B有上行数据,即为主播;C是观众用户,不需要进行数据推流。即该过程包括:
1101、接口机获取用户的类型。
1102、接口机根据用户类型分配端口,以与适配模块进行连接。
1103、适配模块接收到A、B各自发送的上行音视频数据后,对数据包排序及丢包恢复,之后将解码出的音频帧逐帧传给音频模块,并且附带音频参数如音频格式、声道数等,将解码出的视频帧逐帧传递给视频模块,并且附带编码格式、分配率等视频参数。适配模块需要接收处理不同协议的数据,如标准的rtp数据,或私有的udp音视频数据。音频模块收到音频数据后,依据保存的推流参数做混音、转码处理,之后发送到推流模块。视频模块接收到视频数据后,依据保存的推流参数做画面布局,并按直播需要的分辨率、码率、帧率进行混流处理,之后发送到推流模块。
1104、推流模块收到音视频数据后将其封装成rtmp格式,推流到直播***。
通过上述实施例可见,本申请提供的网络直播数据管理的方法可以将实时多人音视频通话的产品同时推流到直播***,使用直播服务。且在直播业务中,也可以采用实时多人通话的客户端来做直播的采集和推流,以获取更佳的推流延迟和弱网推流效果。推流可灵活定制分辨率、码率、帧率、画面 布局。
上述实施例中的方法和装置具体应用于多终端媒体内容融合的网络直播场景中,在该场景中,用户可以灵活的加入多终端媒体内容的交互过程,例如:多人视频通话的过程,然后将该视频通话内容推流至网络直播***,以便于对网络直播过程中的媒体内容进行更新。如图12所示,为本申请实施例提供的另一种网络直播数据的管理方法的流程图,图中第一终端为准备加入直播间并进行媒体内容推流的数据来源,该直播间中第二终端的媒体内容正在推流至第三终端进行播放,其中,第一终端和第二终端可以通过客户端SDK生成网络直播媒体内容,第三终端可以通过H5页面收看网络直播。具体的,上述过程包括如下步骤:
1201、第二终端与第三终端建立了媒体内容的第一连接。
本实施例中,第二终端与第三终端建立的第一连接即第三终端正在播放第二终端推送的媒体内容。
1202、第一终端向第二终端发送加入请求。
本实施例中,第一终端向第二终端发送的加入请求可以包括第一终端的推流参数,即第一终端生成的媒体内容的特征参数,例如:画面分辨率、码率、帧率、画面布局等。
1203、第二终端向控制服务器申请混流。
1204、第二终端确定适配的目标端口。
1205、第二终端与第一终端进行配置并同步目标端口,以建立第二连接。
1206、第二终端向第三终端发送推流数据。
1207、第一终端、第二终端与第三终端之间建立媒体内容第三连接。
本实施例中,步骤1203-步骤1207的内容与图4所示实施例的步骤404-步骤413类似,相关特征可以进行参考,此处不做赘述。
1208、第三终端根据推流数据进行媒体内容的播放。
在一种可能的场景中,第三终端的页面变化情况如图13所示,图13为本申请实施例提供的一种网络直播页面的更新示意图,图中左侧为第三终端与第二终端建立第一连接时的直播界面,在经过上述实施例中第一终端的媒体内容加入后,由于在推流数据中包括了页面布局参数,直播界面可以进行重新排布,即得到如图13右侧的直播界面,从而实现了多终端媒体内容的融合过程并推流至直播***播放,提高了直播页面的媒体内容的丰富度,且在添加媒体内容时具有良好的稳定性。
本申请实施例还提供了一种终端设备,如图14所示,是本申请实施例提供的另一种终端设备的结构示意图,为了便于说明,仅示出了与本申请实施例相关的部分,具体技术细节未揭示的,请参照本申请实施例方法部分。该 终端可以为包括手机、平板电脑、个人数字助理(personal digital assistant,PDA)、销售终端(point of sales,POS)、车载电脑等任意终端设备,以终端为手机为例:
图14示出的是与本申请实施例提供的终端相关的手机的部分结构的框图。参考图14,手机包括:射频(radio frequency,RF)电路1410、存储器1420、输入单元1430、显示单元1440、传感器1450、音频电路1460、无线保真(wireless fidelity,WiFi)模块1470、处理器1480、以及电源1490等部件。本领域技术人员可以理解,图14中示出的手机结构并不构成对手机的限定,可以包括比图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。
下面结合图14对手机的各个构成部件进行具体的介绍:
RF电路1410可用于收发信息或通话过程中,信号的接收和发送,特别地,将基站的下行信息接收后,给处理器1480处理;另外,将设计上行的数据发送给基站。通常,RF电路1410包括但不限于天线、至少一个放大器、收发信机、耦合器、低噪声放大器(low noise amplifier,LNA)、双工器等。此外,RF电路1410还可以通过无线通信与网络和其他设备通信。上述无线通信可以使用任一通信标准或协议,包括但不限于全球移动通讯***(global system of mobile communication,GSM)、通用分组无线服务(general packet radio service,GPRS)、码分多址(code division multiple access,CDMA)、宽带码分多址(wideband code division multiple access,WCDMA)、长期演进(long term evolution,LTE)、电子邮件、短消息服务(short messaging service,SMS)等。
存储器1420可用于存储软件程序以及模块,处理器1480通过运行存储在存储器1420的软件程序以及模块,从而执行手机的各种功能应用以及数据处理。存储器1420可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作***、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等;存储数据区可存储根据手机的使用所创建的数据(比如音频数据、电话本等)等。此外,存储器1420可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
输入单元1430可用于接收输入的数字或字符信息,以及产生与手机的用户设置以及功能控制有关的键信号输入。具体地,输入单元1430可包括触控面板1431以及其他输入设备1432。触控面板1431,也称为触摸屏,可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板1431上或在触控面板1431附近的操作,以及在触控面板1431上一定范围内的隔空触控操作),并根据预先设定的程式驱动相应的连接装置。可选的,触控面板1431可包括触摸检测装置和触摸控制器两个部分。 其中,触摸检测装置检测用户的触摸方位,并检测触摸操作带来的信号,将信号传送给触摸控制器;触摸控制器从触摸检测装置上接收触摸信息,并将它转换成触点坐标,再送给处理器1480,并能接收处理器1480发来的命令并加以执行。此外,可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板1431。除了触控面板1431,输入单元1430还可以包括其他输入设备1432。具体地,其他输入设备1432可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
显示单元1440可用于显示由用户输入的信息或提供给用户的信息以及手机的各种菜单。显示单元1440可包括显示面板1441,可选的,可以采用液晶显示器(liquid crystal display,LCD)、有机发光二极管(organic light-emitting diode,OLED)等形式来配置显示面板1441。进一步的,触控面板1431可覆盖显示面板1441,当触控面板1431检测到在其上或附近的触摸操作后,传送给处理器1480以确定触摸事件的类型,随后处理器1480根据触摸事件的类型在显示面板1441上提供相应的视觉输出。虽然在图14中,触控面板1431与显示面板1441是作为两个独立的部件来实现手机的输入和输入功能,但是在某些实施例中,可以将触控面板1431与显示面板1441集成而实现手机的输入和输出功能。
手机还可包括至少一种传感器1450,比如光传感器、运动传感器以及其他传感器。具体地,光传感器可包括环境光传感器及接近传感器,其中,环境光传感器可根据环境光线的明暗来调节显示面板1441的亮度,接近传感器可在手机移动到耳边时,关闭显示面板1441和/或背光。作为运动传感器的一种,加速计传感器可检测各个方向上(一般为三轴)加速度的大小,静止时可检测出重力的大小及方向,可用于识别手机姿态的应用(比如横竖屏切换、相关游戏、磁力计姿态校准)、振动识别相关功能(比如计步器、敲击)等;至于手机还可配置的陀螺仪、气压计、湿度计、温度计、红外线传感器等其他传感器,在此不再赘述。
音频电路1460、扬声器1461,传声器1462可提供用户与手机之间的音频接口。音频电路1460可将接收到的音频数据转换后的电信号,传输到扬声器1461,由扬声器1461转换为声音信号输出;另一方面,传声器1462将收集的声音信号转换为电信号,由音频电路1460接收后转换为音频数据,再将音频数据输出处理器1480处理后,经RF电路1410以发送给比如另一手机,或者将音频数据输出至存储器1420以便进一步处理。
WiFi属于短距离无线传输技术,手机通过WiFi模块1470可以帮助用户收发电子邮件、浏览网页和访问流式媒体等,它为用户提供了无线的宽带互联网访问。虽然图14示出了WiFi模块1470,但是可以理解的是,其并不属于手机的必须构成,完全可以根据需要在不改变发明的本质的范围内而省略。
处理器1480是手机的控制中心,利用各种接口和线路连接整个手机的各个部分,通过运行或执行存储在存储器1420内的软件程序和/或模块,以及调用存储在存储器1420内的数据,执行手机的各种功能和处理数据,从而对手机进行整体监控。可选的,处理器1480可包括一个或多个处理单元;可选的,处理器1480可集成应用处理器和调制解调处理器,其中,应用处理器主要处理操作***、用户界面和应用程序等,调制解调处理器主要处理无线通信。可以理解的是,上述调制解调处理器也可以不集成到处理器1480中。
手机还包括给各个部件供电的电源1490(比如电池),可选的,电源可以通过电源管理***与处理器1480逻辑相连,从而通过电源管理***实现管理充电、放电、以及功耗管理等功能。
尽管未示出,手机还可以包括摄像头、蓝牙模块等,在此不再赘述。
在本申请实施例中,该终端所包括的处理器1480还具有执行如上述网络直播数据的管理方法的各个步骤的功能。
本申请实施例还提供了一种控制服务器,请参阅图15,图15是本申请实施例提供的另一种控制服务器的结构示意图,该控制服务器1500可因配置或性能不同而产生比较大的差异,可以包括一个或一个以***处理器(central processing units,CPU)1522(例如,一个或一个以上处理器)和存储器1532,一个或一个以上存储应用程序1542或数据1544的存储介质1530(例如一个或一个以上海量存储设备)。其中,存储器1532和存储介质1530可以是短暂存储或持久存储。存储在存储介质1530的程序可以包括一个或一个以上模块(图示没标出),每个模块可以包括对服务器中的一系列指令操作。更进一步地,中央处理器1522可以设置为与存储介质1530通信,在服务器1500上执行存储介质1530中的一系列指令操作。
服务器1500还可以包括一个或一个以上电源1526,一个或一个以上有线或无线网络接口1550,一个或一个以上输入输出接口1558,和/或,一个或一个以上操作***1541,例如Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM等等。
上述实施例中由网络直播数据的管理装置所执行的步骤可以基于该图15所示的服务器结构。
本申请实施例中还提供一种计算机可读存储介质,该计算机可读存储介质中存储有数据管理指令,当其在计算机上运行时,使得计算机执行如前述图3至图4所示实施例描述的方法中多点控制设备或数据接入设备所执行的步骤。
本申请实施例中还提供一种包括数据管理指令的计算机程序产品,当其在计算机上运行时,使得计算机执行如前述图3至图4所示实施例描述的方法中多点控制设备或数据接入设备所执行的步骤。
本申请实施例还提供了一种数据管理***,所述数据管理***可以包含图5所描述实施例中的控制服务器,或者图6所描述的网络设备。
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的***,装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。
在本申请所提供的几个实施例中,应该理解到,所揭露的***,装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个***,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,数据管理装置,或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(read-only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述,以上实施例仅用以说明本申请的技术方案,而非对其限制;尽管参照前述实施例对本申请进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。
Claims (16)
- 一种网络直播数据的管理方法,由多点控制设备执行,所述方法包括:接收数据接入设备发送的推流请求,所述推流请求包括推流参数,所述推流参数是所述数据接入设备基于待发送的推流数据的媒体内容特征确定的,所述推流数据包括来自至少二个数据来源的媒体内容数据;根据所述推流参数,确定适配于所述媒体内容特征的目标端口;通过所述目标端口接收所述数据接入设备发送的所述推流数据;向直播设备推送所述推流数据,以使所述直播设备在进行网络直播过程中播放所述媒体内容数据。
- 根据权利要求1所述的方法,在接收所述数据接入设备发送的推流请求之前,所述方法还包括:按照预设周期获取所述多点控制设备的关联设备的负载信息,所述负载信息用于指示所述关联设备的异常情况;若所述负载信息不满足预设条件,则根据所述负载信息确定发生异常的所述关联设备的地址信息,所述预设条件是基于所述关联设备与所述多点控制设备的连接情况以及所述关联设备的负载情况确定的;根据所述地址信息进行所述关联设备的地址切换,以使所述关联设备的负载信息满足所述预设条件。
- 根据权利要求1所述的方法,根据所述推流参数,确定适配于所述媒体内容特征的目标端口,包括:接收所述数据接入设备发送的混流请求;根据所述混流请求确定空闲端口集合;根据所述推流参数在所述空闲端口集合中选取所述目标端口。
- 根据权利要求1所述的方法,向直播设备推送所述推流数据,包括:确定所述推流数据的媒体内容类型;根据所述媒体内容类型对所述推流数据进行分类,确定所述推流数据的分类类型;将所述推流数据转换成与所述分类类型对应的目标格式,并推送至所述直播设备。
- 根据权利要求1所述的方法,所述方法还包括:接收所述直播设备发送的目标参数,所述目标参数是基于所述网络直播过程中的媒体内容特征确定的;根据所述目标参数对所述推流数据进行调整。
- 根据权利要求1所述的方法,所述方法还包括:接收所述数据接入设备发送的页面布局参数,所述页面布局参数用于指示所述数据接入设备中来自所述至少二个数据来源的媒体内容数据的页面 分布情况;根据所述页面布局参数对所述推流数据进行调整。
- 根据权利要求1-6任一项所述的方法,所述方法还包括:获取策略调度设备发送的目标密钥,所述目标密钥用于指示所述直播设备的地址;根据所述目标密钥对所述推流数据进行加密。
- 一种网络直播数据的管理方法,由数据接入设备执行,所述方法包括:接收至少二个数据来源发送的推流数据;根据所述推流数据的媒体内容特征确定推流参数;向多点控制设备发送所述推流参数,所述推流参数用于指示所述多点控制设备确定适配于所述媒体内容特征的目标端口;接收所述多点控制设备发送的所述目标端口的信息;通过所述目标端口发送所述推流数据。
- 根据权利要求8所述的方法,所述方法还包括:根据所述推流数据向所述多点控制设备发送混流请求,所述混流请求用于指示所述多点控制设备确定空闲端口集合;接收所述多点控制设备发送的所述空闲端口集合;根据所述空闲端口集合进行端口同步。
- 根据权利要求8所述的方法,所述方法还包括:获取所述数据来源的用户类型;根据所述用户类型确定接入所述目标端口的数据来源。
- 根据权利要求8所述的方法,所述方法还包括:获取所述数据来源发送的页面布局参数;向所述多点控制设备发送所述页面布局参数,以使所述多点控制设备根据所述页面布局参数对所述推流数据进行调整。
- 一种控制服务器,所述控制服务器包括:调度模块,用于接收数据接入设备发送的推流请求,所述推流请求包括推流参数,所述推流参数是所述数据接入设备基于待发送的推流数据的媒体内容特征确定的,所述推流数据包括来自至少二个数据来源的媒体内容数据;适配模块,用于根据所述推流参数,确定适配于所述媒体内容特征的目标端口;媒体内容模块,用于通过所述目标端口接收所述数据接入设备发送的所述推流数据;推流模块,用于向直播设备推送所述推流数据,以使所述直播设备在进行网络直播过程中播放所述媒体内容数据。
- 一种网络设备,所述网络设备包括:接入模块,用于接收至少二个数据来源发送的推流数据;流控模块,用于根据所述推流数据的媒体内容特征确定推流参数;所述流控模块,还用于向多点控制设备发送所述推流参数,所述推流参数用于指示所述多点控制设备确定适配于所述媒体内容特征的目标端口;所述流控模块,还用于接收所述多点控制设备发送的所述目标端口的信息;所述接入模块,还用于通过所述目标端口发送所述推流数据。
- 一种计算机设备,所述计算机设备包括处理器以及存储器:所述存储器用于存储程序代码;所述处理器用于根据所述程序代码中的指令执行权利要求1至7任一项所述的网络直播数据的管理方法,或权利要求8至11任一项所述的网络直播数据的管理方法。
- 一种计算机可读存储介质,所述计算机可读存储介质中存储有指令,当其在计算机上运行时,使得计算机执行上述权利要求1至7任一项所述的网络直播数据的管理方法,或权利要求8至11任一项所述的网络直播数据的管理方法。
- 一种计算机程序产品,包括指令,当其在计算机上运行时,使得计算机执行如权利要求1至7任一项所述的网络直播数据的管理方法,或权利要求8至11任一项所述的网络直播数据的管理方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/693,518 US11936921B2 (en) | 2020-02-20 | 2022-03-14 | Method for managing network live streaming data and related apparatus, and device and storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010104378.1 | 2020-02-20 | ||
CN202010104378.1A CN111225230B (zh) | 2020-02-20 | 2020-02-20 | 一种网络直播数据的管理方法以及相关装置 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/693,518 Continuation US11936921B2 (en) | 2020-02-20 | 2022-03-14 | Method for managing network live streaming data and related apparatus, and device and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021164352A1 true WO2021164352A1 (zh) | 2021-08-26 |
Family
ID=70831357
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/130005 WO2021164352A1 (zh) | 2020-02-20 | 2020-11-19 | 一种网络直播数据的管理方法以及相关装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US11936921B2 (zh) |
CN (1) | CN111225230B (zh) |
WO (1) | WO2021164352A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113727183A (zh) * | 2021-08-30 | 2021-11-30 | 北京百度网讯科技有限公司 | 直播推流方法、装置、设备、存储介质及计算机程序产品 |
CN114598897A (zh) * | 2022-02-28 | 2022-06-07 | 阿里巴巴(中国)有限公司 | 直播调度方法、设备和可读介质 |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111225230B (zh) | 2020-02-20 | 2022-10-04 | 腾讯科技(深圳)有限公司 | 一种网络直播数据的管理方法以及相关装置 |
CN111669661B (zh) * | 2020-06-23 | 2021-06-22 | 四川长虹电器股份有限公司 | 一种基于trtc协议的电视音视频通话***及方法 |
CN113747181A (zh) * | 2020-08-06 | 2021-12-03 | 上海达龙信息科技有限公司 | 基于远程桌面的网络直播方法、直播***及电子设备 |
CN114422863B (zh) * | 2020-10-13 | 2023-05-12 | 上海哔哩哔哩科技有限公司 | 视频互动直播方法及*** |
CN112511858B (zh) * | 2020-10-26 | 2023-04-07 | 深圳点猫科技有限公司 | 一种网课直播混合推流方法、***及存储介质 |
CN112437322B (zh) * | 2020-11-12 | 2023-05-09 | 创盛视联数码科技(北京)有限公司 | 数据处理方法、装置、多媒体***、电子设备及存储介质 |
CN112911327B (zh) * | 2021-02-01 | 2023-03-28 | 深圳市茅庐信息科技有限公司 | 一种直播软件自动配置方法、***及存储介质 |
CN116055658B (zh) * | 2023-01-10 | 2024-06-04 | 北京市博汇科技股份有限公司 | 网站及app的多画面实时图像帧融合监测方法及*** |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020015092A1 (en) * | 2000-01-13 | 2002-02-07 | Meir Feder | Method and system for multimedia communication control |
CN1810028A (zh) * | 2003-04-17 | 2006-07-26 | 西门子通讯公司 | 用于实时回放会议流的***和方法 |
CN103139217A (zh) * | 2013-02-21 | 2013-06-05 | 中国对外翻译出版有限公司 | 基于sip的多方通话实现方法 |
CN105141971A (zh) * | 2015-09-16 | 2015-12-09 | 深圳市前海智媒网络科技有限公司 | 一种基于会话初始化协议实现直播的方法及*** |
CN105635636A (zh) * | 2015-12-30 | 2016-06-01 | 随锐科技股份有限公司 | 一种视频会议***及其实现视频图像传输控制的方法 |
CN106330753A (zh) * | 2015-06-26 | 2017-01-11 | 中兴通讯股份有限公司 | 流处理方法和装置 |
CN106803923A (zh) * | 2017-03-01 | 2017-06-06 | 北京牡丹电子集团有限责任公司数字电视技术中心 | Rtmp推流实时直播*** |
CN109361893A (zh) * | 2018-11-21 | 2019-02-19 | 华平信息技术股份有限公司 | 视频会议同路多源直播方法及*** |
CN111225230A (zh) * | 2020-02-20 | 2020-06-02 | 腾讯科技(深圳)有限公司 | 一种网络直播数据的管理方法以及相关装置 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE60238100D1 (de) * | 2001-09-14 | 2010-12-09 | Polycom Israel Ltd | Vorrichtung und Verfahren für Multimedia- Videoverarbeitung |
US20060013212A1 (en) * | 2004-07-13 | 2006-01-19 | Hartej Singh | Port aggregation across stack of devices |
US20090219992A1 (en) * | 2005-08-04 | 2009-09-03 | Charles Chunaming Wang | Compensating delay of channel state information between receiver and transmitter during adaptive video delivery |
US8935733B2 (en) * | 2006-09-07 | 2015-01-13 | Porto Vinci Ltd. Limited Liability Company | Data presentation using a wireless home entertainment hub |
US8005227B1 (en) * | 2007-12-27 | 2011-08-23 | Emc Corporation | Key information consistency checking in encrypting data storage system |
CA2791870C (en) * | 2010-03-05 | 2017-01-17 | General Instrument Corporation | Method and apparatus for converting two-dimensional video content for insertion into three-dimensional video content |
US8994778B2 (en) * | 2011-12-07 | 2015-03-31 | Positron Telecommunication Systems, Inc. | Systems and methods for providing video conferencing services via an ethernet adapter |
CN107396206B (zh) * | 2017-09-05 | 2021-08-13 | 广州华多网络科技有限公司 | 一种直播数据的推流方法及*** |
US11240280B2 (en) * | 2019-02-19 | 2022-02-01 | Apple Inc. | Low latency streaming media |
US11119985B1 (en) * | 2021-03-19 | 2021-09-14 | Atlassian Pty Ltd. | Apparatuses, methods, and computer program products for the programmatic documentation of extrinsic event based data objects in a collaborative documentation service |
-
2020
- 2020-02-20 CN CN202010104378.1A patent/CN111225230B/zh active Active
- 2020-11-19 WO PCT/CN2020/130005 patent/WO2021164352A1/zh active Application Filing
-
2022
- 2022-03-14 US US17/693,518 patent/US11936921B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020015092A1 (en) * | 2000-01-13 | 2002-02-07 | Meir Feder | Method and system for multimedia communication control |
CN1810028A (zh) * | 2003-04-17 | 2006-07-26 | 西门子通讯公司 | 用于实时回放会议流的***和方法 |
CN103139217A (zh) * | 2013-02-21 | 2013-06-05 | 中国对外翻译出版有限公司 | 基于sip的多方通话实现方法 |
CN106330753A (zh) * | 2015-06-26 | 2017-01-11 | 中兴通讯股份有限公司 | 流处理方法和装置 |
CN105141971A (zh) * | 2015-09-16 | 2015-12-09 | 深圳市前海智媒网络科技有限公司 | 一种基于会话初始化协议实现直播的方法及*** |
CN105635636A (zh) * | 2015-12-30 | 2016-06-01 | 随锐科技股份有限公司 | 一种视频会议***及其实现视频图像传输控制的方法 |
CN106803923A (zh) * | 2017-03-01 | 2017-06-06 | 北京牡丹电子集团有限责任公司数字电视技术中心 | Rtmp推流实时直播*** |
CN109361893A (zh) * | 2018-11-21 | 2019-02-19 | 华平信息技术股份有限公司 | 视频会议同路多源直播方法及*** |
CN111225230A (zh) * | 2020-02-20 | 2020-06-02 | 腾讯科技(深圳)有限公司 | 一种网络直播数据的管理方法以及相关装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113727183A (zh) * | 2021-08-30 | 2021-11-30 | 北京百度网讯科技有限公司 | 直播推流方法、装置、设备、存储介质及计算机程序产品 |
CN113727183B (zh) * | 2021-08-30 | 2023-09-15 | 北京百度网讯科技有限公司 | 直播推流方法、装置、设备、存储介质及计算机程序产品 |
CN114598897A (zh) * | 2022-02-28 | 2022-06-07 | 阿里巴巴(中国)有限公司 | 直播调度方法、设备和可读介质 |
Also Published As
Publication number | Publication date |
---|---|
CN111225230B (zh) | 2022-10-04 |
US11936921B2 (en) | 2024-03-19 |
CN111225230A (zh) | 2020-06-02 |
US20220210485A1 (en) | 2022-06-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021164352A1 (zh) | 一种网络直播数据的管理方法以及相关装置 | |
US20200351562A1 (en) | System and method for multi-user digital interactive experience | |
US9591262B2 (en) | Flow-control based switched group video chat and real-time interactive broadcast | |
CN103368935B (zh) | 在Wi-Fi显示网络中提供增强Wi-Fi显示会话的方法和装置 | |
US20170134831A1 (en) | Flow Controlled Based Synchronized Playback of Recorded Media | |
US8880098B2 (en) | Communication server and session control method | |
WO2018184266A1 (zh) | 直播频道中的通信方法、装置及*** | |
US20120056971A1 (en) | Virtual Presence Via Mobile | |
US20140331135A1 (en) | Digital content connectivity and control via a plurality of controllers that are treated as a single controller | |
US20120066722A1 (en) | Enhanced Video Sharing | |
US9357215B2 (en) | Audio output distribution | |
US20140344854A1 (en) | Method and System for Displaying Speech to Text Converted Audio with Streaming Video Content Data | |
US11889159B2 (en) | System and method for multi-user digital interactive experience | |
CN103067776A (zh) | 节目推送方法、***及智能显示设备、云端服务器 | |
CN108924603B (zh) | 基于软件定义WiFi的多终端流媒体播放***和方法 | |
US9832422B2 (en) | Selective recording of high quality media in a videoconference | |
EP3306947A1 (en) | Video processing device, method for driving video processing device, video relay device, method for driving video relay device, and computer-readable recording medium | |
US11431770B2 (en) | Method, system, apparatus, and electronic device for managing data streams in a multi-user instant messaging system | |
JP2021527351A (ja) | ネットワーク容量に制約のあるシナリオにおける共同メディア制作のためのネットワーク制御上りリンクメディア伝送 | |
US10122896B2 (en) | System and method of managing transmission of data between two devices | |
JP6396342B2 (ja) | オーディオ−ビデオ用のワイヤレスドッキングシステム | |
WO2024017296A1 (zh) | 共享方法、电子设备及*** | |
JP2017028646A (ja) | 中継装置、通信システム、中継方法、及び中継プログラム | |
Zhiyuan et al. | A Cloud-Based Pan-Terminal Video Conferencing System | |
JP2014078180A (ja) | サーバ装置、情報配信システム及び情報配信プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20920416 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 09/12/2022) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20920416 Country of ref document: EP Kind code of ref document: A1 |