CN110784757A - Game live broadcast method and device in live broadcast client, medium and electronic equipment - Google Patents
Game live broadcast method and device in live broadcast client, medium and electronic equipment Download PDFInfo
- Publication number
- CN110784757A CN110784757A CN201911071870.7A CN201911071870A CN110784757A CN 110784757 A CN110784757 A CN 110784757A CN 201911071870 A CN201911071870 A CN 201911071870A CN 110784757 A CN110784757 A CN 110784757A
- Authority
- CN
- China
- Prior art keywords
- live
- game
- live broadcast
- client
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000009877 rendering Methods 0.000 claims abstract description 17
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 15
- 238000010586 diagram Methods 0.000 description 9
- AWSBQWZZLBPUQH-UHFFFAOYSA-N mdat Chemical compound C1=C2CC(N)CCC2=CC2=C1OCO2 AWSBQWZZLBPUQH-UHFFFAOYSA-N 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 238000003491 array Methods 0.000 description 2
- 238000005538 encapsulation Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 208000003028 Stuttering Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4335—Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4781—Games
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
- H04N21/8173—End-user applications, e.g. Web browser, game
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a game live broadcast method in a live broadcast client, a game live broadcast device in the live broadcast client, a computer readable storage medium and electronic equipment, and relates to the technical field of video live broadcast. In the game live broadcasting method in the live broadcasting client, the live broadcasting client runs on an electronic terminal, and the electronic terminal comprises a game client, wherein the game live broadcasting method in the live broadcasting client comprises the following steps: the method comprises the steps of obtaining live broadcast images of a game through an image rendering thread of a game client, obtaining live broadcast audio of the game, coding the live broadcast images and the live broadcast audio, and pushing the coded live broadcast images and the live broadcast audio to a live broadcast server. The method and the device can reduce the jamming problem of the game and improve the live broadcast processing efficiency in the live broadcast process.
Description
Technical Field
The present disclosure relates to the field of video live broadcast technologies, and in particular, to a game live broadcast method in a live broadcast client, a game live broadcast device in the live broadcast client, a computer-readable storage medium, and an electronic device.
Background
With the progress of internet technology, live webcasts are also rapidly developed and widely applied. For example, large game manufacturers combine gaming with live broadcasting to enhance the promotion and promotion of game products.
At present, when a game is played directly, two technical schemes are adopted, wherein the first scheme comprises the following steps: and recording game pictures of the game client by using third-party screen recording software, and pushing the recorded pictures to a server for image processing to obtain live video. The second scheme is as follows: the live broadcast device is embedded into the game processor, and the game processor processes the pictures of the game pictures and then pushes the processed game pictures to the live broadcast device. However, the two existing technical solutions can cause the problems of complex live broadcast operation process, game pause and the like.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to a game live broadcasting method in a live broadcasting client, a game live broadcasting device in a live broadcasting client, a computer-readable storage medium, and an electronic device, thereby overcoming, at least to some extent, a problem of game stutter due to limitations and disadvantages of related art.
According to a first aspect of the present disclosure, a game live broadcast method in a live broadcast client is provided, where the live broadcast client operates in an electronic terminal, the electronic terminal includes a game client, and the game live broadcast method in the live broadcast client includes: acquiring a live broadcast image of a game through an image rendering thread of a game client; acquiring live broadcast audio of a game; and coding and pushing the live images and the live audio to a live server.
According to a second aspect of the present disclosure, there is provided a game live broadcasting device in a live broadcasting client, including: the image acquisition module is used for acquiring a live image of the game through an image rendering thread of the game client; the audio acquisition module is used for acquiring live audio of the game; and the data coding module is used for coding the live image and the live audio and pushing the coded live audio to a live server.
Optionally, the data encoding module comprises: the image coding unit is used for coding the live broadcast image to generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data; the video data generating unit is used for extracting image data from the streaming media file and generating target video data corresponding to the image data if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file; the audio data generating unit is used for coding the live broadcast audio to generate target audio data; and the data processing unit is used for processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to the live broadcast server.
Alternatively, the image encoding unit may be configured to perform: and coding the live broadcast image based on the preset video resolution to generate a streaming media file.
Optionally, the image acquisition module may be configured to perform: and acquiring a live image of the game from an image rendering thread of the game client based on a preset video frame rate.
Alternatively, the video data generation unit may be configured to perform: and extracting image data from the streaming media file according to a preset video frame rate, and generating target video data corresponding to the image data.
Alternatively, the image encoding unit may be configured to perform: and zooming the live broadcast image, and encoding the zoomed live broadcast image.
Optionally, the game live broadcasting device in the live client further includes: and the information adding module is used for adding protection information to the coded live image.
Alternatively, the image encoding unit may be configured to perform: and carrying out zooming processing on the live image, and carrying out coding and protection information adding processing on the zoomed live image.
Optionally, the game live broadcasting device in the live client further includes: the storage space determining module is used for determining the available storage space of a file library for storing the streaming media file; and the file library emptying module is used for emptying the file library and recoding the direct-broadcast image to generate the streaming media file to be stored in the file library if the streaming media file is larger than the available storage space.
Optionally, the game live broadcasting device in the live client further includes: the encoding duration determining module is used for determining the encoding duration for encoding the live image; and the coding duration judging module is used for recoding the direct broadcast image if the coding duration is greater than a preset threshold.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements a game live method in a live client as described above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: one or more processors; a storage device for storing one or more programs which, when executed by one or more processors, cause the one or more processors to implement a method of live gaming in a live client as described above.
Exemplary embodiments of the present disclosure have the following advantageous effects:
in some embodiments of the present disclosure, a live client operates in an electronic terminal, and the electronic terminal includes a game client. Specifically, a live broadcast client acquires a live broadcast image of a game through an image rendering thread of a game client; acquiring live broadcast audio of a game; and coding and pushing the live images and the live audio to a live server. On the one hand, the live broadcast client can acquire live broadcast images of games from the game client, so that the processing steps of pushing the live broadcast images for the live broadcast client by the game client are reduced, the live broadcast images are not acquired by using third-party recording software, the operation in the live broadcast process is simplified, and the live broadcast processing efficiency in the live broadcast process is improved. On the other hand, the live broadcast client encodes the live broadcast images and the live broadcast audio, so that the process of encoding and streaming the images through the game client in the condition that the live broadcast client is embedded into the game client is avoided, the consumption of game client resources is reduced, the pause of a game is reduced, and the smoothness of the game is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
fig. 1 schematically illustrates a flow chart of a game live method in a live client according to an exemplary embodiment of the present disclosure;
FIG. 2 schematically illustrates an interaction flow diagram of a game live method in a live client according to an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a block diagram of a gaming live device in a live client according to an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a block diagram of a data encoding module according to an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a gaming live device in a live client according to another exemplary embodiment of the present disclosure;
fig. 8 schematically shows a block diagram of an electronic device in an exemplary embodiment according to the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In the present disclosure, the terms "comprises" and "comprising" are used in an open-ended fashion, and mean that there may be additional elements/components/etc. in addition to the listed elements/components/etc.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
For ease of understanding, the scenario to which the present solution relates will be explained first.
With the progress of internet technology, live webcasts are rapidly developed and applied, for example, live games.
In the live game, the anchor firstly opens third-party screen recording software to collect screen pictures containing game content pictures, then preprocesses the recorded screen pictures to extract the game content pictures, then pushes the processed game content pictures to a live broadcast server, and then sends the processed game content pictures to a user for watching by the live broadcast server. However, in the above process, the game software and the third-party screen recording software are incompatible, complicated to operate and the like, and the game jamming problem is also caused. In response to these problems, the present disclosure provides a method for live game play in a live client.
It should be noted that, in the exemplary embodiment of the present disclosure, the game live method in the live client described below may be generally implemented by the live client, that is, each step of the game live method in the live client may be executed by the live client, in which case, the game live device in the live client may be configured in the live client.
In addition, the game live broadcasting method in the live client described below may be generally implemented by a terminal device (e.g., a mobile phone, a tablet, a personal computer, etc.), that is, the steps of the game live broadcasting method in the live client may be performed by the terminal device, in which case, the game live broadcasting device in the live client may be configured in the terminal device.
Hereinafter, the steps of the game live broadcast method in the live broadcast client in the present exemplary embodiment will be described in more detail with reference to the drawings and the examples.
Fig. 1 schematically shows a flowchart of a game live method in a live client according to an exemplary embodiment of the present disclosure. In the following description, a live client is taken as an execution subject for illustration, the live client runs on an electronic terminal, and the electronic terminal includes a game client. Referring to fig. 1, a game live broadcasting method in a live client may include the steps of:
and S102, acquiring a live broadcast image of the game through an image rendering thread of the game client.
In an exemplary embodiment of the present disclosure, a game image is acquired from a game in which a game client is running to be live-played, wherein the game image is a live-played image of the game in the present exemplary embodiment. The image rendering thread of the game client can render each frame of image in the game. The live image of the game may contain the picture content of the game. It should be noted that, in the exemplary embodiment of the present disclosure, the type of game executed in the game client is not limited, for example, a shooting game, a fighting game, and the like.
According to one embodiment of the present disclosure, the live client may set a preset video frame rate to acquire live images, where the preset video frame rate may be set to 20fps, may be set to 30fps, and so on. In addition, the live broadcast client can also set a plurality of preset video frame rates, and display the plurality of preset video frame rates to the main broadcast in a mode of acquiring the speed gear of the live broadcast image.
That is, in an exemplary embodiment of the present disclosure, the live client may acquire a live image of the game from an image rendering thread of the game client based on a preset video frame rate. For example, the anchor may obtain a game image from an image rendering thread of the game client based on a preset video frame rate of 20 fps.
After the live image is acquired, the live client may also perform scaling processing on the acquired live image. In addition, in order to protect the game from being copied, the live client may add protection information to the game image after acquiring the game image, where the protection information may show an identifier of the anchor in the form of a watermark, and the identifier of the anchor may contain a nickname of the anchor, an anchor number of the anchor, and so on. The live broadcast client can also perform processing of adding protection information after performing scaling processing on live broadcast images.
The live broadcast client side of the embodiment can acquire live broadcast images of games, processing of the game client side on the live broadcast images is reduced, the fact that third-party recording software is used for acquiring the live broadcast images is avoided, operation in a live broadcast process is simplified, and live broadcast cost is reduced. Meanwhile, the live client adds the protection information to visually display the ownership of the live video of the anchor broadcast, so that the user can conveniently inquire and understand the ownership.
And S104, acquiring live audio of the game.
The live audio of the game may comprise audio in the game and may also comprise audio for the main lecture game. It should be noted that the audio of the main broadcasting explanation game can be collected by a third-party recording device and transmitted to the live broadcasting client, or collected by a recording device in the electronic terminal. Live audio of the game may also be captured using microphones in exemplary embodiments of the present disclosure. The microphone may be a microphone device or a set of microphone arrays.
In addition, this example embodiment may also perform preprocessing on the captured live audio of the game, where the preprocessing may include: the sound of the audio of the main broadcasting explaining game is filtered, for example, the sound of the road, the noise contained in the audio of the game in the case of unstable current, and the like.
And S106, coding the live image and the live audio and pushing the coded live image and the live audio to a live server.
In an exemplary embodiment of the present disclosure, a live client may encode live images, encode live audio, and then synchronize the live images and the live audio.
Specifically, according to an embodiment of the present disclosure, a live client may encode a live image to generate a streaming media file; the streaming media file may include identification information of the streaming media file and image data, and the identification information of the streaming media file may include coding information, index information, and the like of the streaming media file; if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file, extracting image data from the streaming media file and generating target video data corresponding to the image data; the identification information of the standard streaming media file may include corresponding standard encoding information and a corresponding data format standard; encoding live audio to generate target audio data; and processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to a live broadcast server.
In an exemplary embodiment of the present disclosure, the streaming media file may be a file in MP4 format, a file in MOV format, a file in M4V format, and the like, and the live client may extract image data from the streaming media file according to a structural composition of the streaming media file. The time stamp may be a current system time stamp from which the image data is extracted as a standard.
Before generating the target video data, the live broadcast client may compare the identification information of the streaming media file with the identification information of the standard streaming media file. If the identification information of the streaming media file is consistent with the identification information of the standard streaming media file, image data can be extracted from the coded streaming media file and packaged into target image data. In addition, the encoding of the live video can be performed according to the existing video encoding technology, or the live video of the game can be written and encoded by the system interface to obtain a streaming media file.
Taking the MP4 streaming media format as an example, the data format of the MP4 file includes moov element and mdat element, where the moov element includes file encoding information, and the mdat element includes image data.
The live broadcast client acquires live broadcast images of the game and writes the live broadcast images into an MP4 streaming media format. Before extracting image data from the written MP4 streaming media format, the live broadcast client firstly extracts moov elements of MP4 streaming media written with live broadcast images, and then judges whether file encoding information contained in the moov elements meets the encoding information of the standard MP4 and whether the data structure meets the standard MP4 streaming media format. If the format of the stream media is consistent with the standard MP4 stream media format, the MP4 stream media written with live images is correct, and the image data can be extracted from the mdat element. It should be noted that, in the case where the streaming media of the MP4 that writes live images has a plurality of mdat elements, the extraction may be performed in mdat units and packaged into VIDEO FLV data (FLASH VIDEO, FLV), that is, target VIDEO data.
In an exemplary embodiment of the present disclosure, a live client encodes live Audio, may encode the live Audio according to an existing Advanced Audio Coding (AAC), and encapsulates the encoded live Audio into Audio FLV data, that is, target Audio data. It should be noted that the method adopted by the encapsulation process of the video FLV data and the audio FLV data may be an encapsulation method in the prior art.
In addition, the target video data and the target audio data are the time stamps and the targets of the target video data analyzed each time according to the system time stamps, and the live broadcast client compares the time stamps of the target video data with the time stamps of the target audio data to perform synchronous processing. That is to say, the live broadcast client may also temporarily store the target audio data and the target video data, and then, based on the timestamp of the target video data and the timestamp of the target audio data, push the target audio data and the target video data to the live broadcast server according to the sequence of the timestamps. The live broadcast client may push target video data and target audio data to a live broadcast server according to a Real Time Message Protocol (RTMP).
According to one embodiment of the present disclosure, a live client may encode a live image based on a preset video resolution to generate a streaming media file. Wherein the preset video resolution may be set to 640 × 320, 720 × 480, etc., and presented to the main broadcast in the form of buttons. In case a live image needs to be encoded, the selection can be made by the anchor.
In addition, the live broadcast client can also extract image data from the streaming media file according to a preset video frame rate and generate target video data corresponding to the image data. For example, in the case where the anchor acquires a live image of a game at a preset video frame rate of 20fps, image data may be extracted according to the same preset video frame rate.
According to the exemplary embodiment of the disclosure, after the live broadcast client performs scaling processing on the live broadcast image, the scaled live broadcast image may be encoded to generate a streaming media file. In addition, the live broadcast client can also add protection information to the live broadcast image, and then encode the live broadcast image added with the protection information to generate a streaming media file. That is, after the live broadcast client performs scaling processing and protection information adding processing on the live broadcast image, the live broadcast image after processing may also be encoded to generate a streaming media file. The live broadcast client carries out zooming processing and protection information adding processing on the live broadcast image, avoids the processes of image zooming and protection information adding executed by the game client in the prior art, reduces the resource waste of the game client, provides more resources for the game and improves the smoothness of the game.
It is noted that in an exemplary embodiment of the present disclosure, the available storage space of a file library storing streaming media files is determined; and if the streaming media file is larger than the available storage space, clearing the file library, and recoding the live image to generate the streaming media file to be stored in the file library. The file library can be arranged in a live client and also can be arranged in an electronic terminal for operating the live client. Due to the limited disk space of the electronic terminal, the live client of the terminal can set the available storage space to 50 MB.
That is, in the case of storing the generated streaming media file, if the size of the streaming media file exceeds the available storage space after the live broadcast client determines that the streaming media file is stored, the current storage step needs to be terminated, the file library needs to be emptied, the live broadcast image needs to be encoded again, and the live broadcast image needs to be stored again.
According to another embodiment of the present disclosure, a live client may determine an encoding duration for encoding a live image; and if the encoding duration is greater than a preset threshold value, re-encoding the live broadcast image. Wherein the preset threshold is calculated according to the existing coding technology.
As shown in fig. 2, an interactive flowchart of a game live method in a live client according to an exemplary embodiment of the present disclosure may be executed in an electronic terminal, where the electronic terminal includes a game client. Specifically, the interaction flow of the game live broadcasting method in the live broadcasting client may include: in step S201, the live broadcast client may obtain a live broadcast image of the game through an image rendering thread of the game client; in step S203, the live client may obtain the audio of the game, and may also obtain the audio of the main broadcasting explaining the game; in step S205, the live client may encode the live image and generate target video data; in step S207, the live client may encode live audio and generate target audio data; in step S209, the live client may push the processed target video data and target audio data to the server.
In an exemplary embodiment of the present disclosure, the live image obtained from the image rendering thread of the game may be in the form of a graphics library Texture unit, e.g., OpenGL Texture.
For example, in the case where the anchor needs to play a live game, the streaming media format of MP4 is adopted.
Firstly, a live broadcast client acquires a live broadcast image of a game at a preset video frame rate of 20fps, and writes the live broadcast image into an MP4 streaming media format; then, storing the streaming media format into a file library with available storage space of 50MB, and if the MP4 streaming media file exceeds 50MB, re-encoding the live image; secondly, before extracting image data from the written MP4 streaming media format, the live client extracts moov elements of MP4 streaming media written with live images; thirdly, whether the file encoding information contained in the moov element conforms to the encoding information of the standard MP4 or not and whether the data structure conforms to the standard MP4 streaming media format or not are judged. If the video data is consistent with the standard MP4 streaming media format, then, the image data is extracted in units of mdat elements, the system time stamp of the extracted image data is recorded, and the image data is packaged into video FLV data.
The live broadcast client collects the audio frequency of the game and the audio frequency of the main broadcast explanation game through a microphone, outputs audio data according to an advanced audio coding algorithm, records a system time stamp corresponding to the output audio data, and encapsulates the audio data into audio FLV data.
And then, the live broadcast client pushes the video FLV data and the audio FLV data to a live broadcast server based on the time stamp of the video FLV data and the time stamp of the audio FLV data so as to send the video FLV data and the audio FLV data to the watching client.
According to the game live broadcasting method in the live broadcasting client, the live broadcasting client is used for acquiring the game live broadcasting image and the live broadcasting audio and encoding the live broadcasting image and the live broadcasting audio, so that encoding processing and packaging processing of the game on the live broadcasting image are avoided, operation steps of the game are reduced, and the game pause is further reduced.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Further, in an exemplary embodiment of the present disclosure, a game live device in a live client is also provided.
Fig. 3 schematically illustrates a block diagram of a game live device in a live client according to an exemplary embodiment of the present disclosure. Referring to fig. 3, a game live device 300 in a live client according to an exemplary embodiment of the present disclosure may include: an image acquisition module 302, an audio acquisition module 304, and a data encoding module 306.
The image obtaining module 302 is configured to obtain a live image of a game through an image rendering thread of a game client; an audio acquisition module 304, configured to acquire live audio of a game; and the data encoding module 306 is configured to encode the live image and the live audio and push the encoded live image and the live audio to a live server.
According to an exemplary embodiment of the present disclosure, referring to fig. 4, the data encoding module 306 may include: an image encoding unit 401, a video data generation unit 403, an audio data generation unit 405, and a data processing unit 407.
The image encoding unit 401 is configured to encode a live image and generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data; a video data generating unit 403, configured to extract image data from the streaming media file and generate target video data corresponding to the image data if the identification information of the streaming media file is consistent with the identification information of the standard streaming media file; an audio data generating unit 405, configured to encode live audio to generate target audio data; and the data processing unit 407 is configured to process the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and push the target video data and the target audio data to the live broadcast server.
According to an exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and coding the live broadcast image based on the preset video resolution to generate a streaming media file.
According to an example embodiment of the present disclosure, the image acquisition module 302 may be configured to perform: and acquiring a live image of the game from an image rendering thread of the game client based on a preset video frame rate.
According to an exemplary embodiment of the present disclosure, the video data generation unit 403 may be configured to perform: and extracting image data from the streaming media file according to a preset video frame rate, and generating target video data corresponding to the image data.
According to another exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and zooming the live broadcast image, and encoding the zoomed live broadcast image.
According to another exemplary embodiment of the present disclosure, referring to fig. 5, compared to the game live device 300 in the live client, the game live device 500 in the live client may further include: an information adding module 502, configured to perform information adding protection processing on the encoded live video.
According to another exemplary embodiment of the present disclosure, the image encoding unit 401 may be configured to perform: and carrying out zooming processing on the live image, and carrying out coding and protection information adding processing on the zoomed live image.
According to another exemplary embodiment of the present disclosure, referring to fig. 6, compared to the game live device 300 in the live client, the game live device 600 in the live client may further include: a storage space determination module 601 and a file library clearing module 603.
The storage space determining module 601 is configured to determine an available storage space of a file library for storing the streaming media file; the file library emptying module 603 is configured to empty the file library if the streaming media file is larger than the available storage space, and re-encode the live image to generate the streaming media file for storage in the file library.
According to another exemplary embodiment of the present disclosure, referring to fig. 7, compared to the game live device 300 in the live client, the game live device 700 in the live client may further include: a code duration determination module 702 and a code duration determination module 704.
The encoding duration determining module 702 is configured to determine an encoding duration for encoding a live image; the encoding duration determining module 704 is configured to re-encode the live image if the encoding duration is greater than a preset threshold.
The details of each module/unit in the above-mentioned apparatus have been described in detail in the embodiments of the method section, and thus are not described again.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when the program product is run on the terminal device.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 800 according to this embodiment of the invention is described below with reference to fig. 8. The electronic device 800 shown in fig. 8 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present invention.
As shown in fig. 8, electronic device 800 is in the form of a general purpose computing device. The components of the electronic device 800 may include, but are not limited to: the at least one processing unit 810, the at least one memory unit 820, a bus 830 connecting different system components (including the memory unit 820 and the processing unit 810), and a display unit 840.
Wherein the storage unit stores program code that is executable by the processing unit 810 to cause the processing unit 810 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 810 may perform steps S102 to S106 shown in fig. 1.
The storage unit 820 may include readable media in the form of volatile memory units such as a random access memory unit (RAM)8201 and/or a cache memory unit 8202, and may further include a read only memory unit (ROM) 8203.
The storage unit 820 may also include a program/utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 800 may also communicate with one or more external devices 900 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to communicate with the electronic device 800, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 800 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 850. Also, the electronic device 800 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 860. As shown, the network adapter 860 communicates with the other modules of the electronic device 800 via the bus 830. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (13)
1. A game live broadcast method in a live broadcast client is characterized in that the live broadcast client runs in an electronic terminal, the electronic terminal comprises a game client, and the game live broadcast method in the live broadcast client comprises the following steps:
acquiring a live broadcast image of the game through an image rendering thread of the game client;
acquiring live broadcast audio of the game;
and coding the live broadcast image and the live broadcast audio and pushing the coded live broadcast image and the live broadcast audio to a live broadcast server.
2. The live game playing method in the live client side according to claim 1, wherein the step of encoding and pushing the live image and the live audio to a live server comprises:
encoding the live broadcast image to generate a streaming media file; the streaming media file comprises identification information of the streaming media file and image data;
if the identification information of the streaming media file is consistent with the identification information of a standard streaming media file, extracting the image data from the streaming media file and generating target video data corresponding to the image data;
encoding the live broadcast audio to generate target audio data;
and processing the target video data and the target audio data according to the timestamp for generating the target video data and the timestamp for generating the target audio data, and pushing the target video data and the target audio data to a live broadcast server.
3. The live game playing method in the live client according to claim 2, wherein encoding the live image and generating a streaming media file comprises:
and coding the live broadcast image based on a preset video resolution ratio to generate a streaming media file.
4. The live game playing method in the live game client according to claim 2 or 3, wherein the step of acquiring the live game image through the image rendering thread of the game client comprises:
and acquiring the live broadcast image of the game from the image rendering thread of the game client based on a preset video frame rate.
5. The game live method in the live client according to claim 4, wherein extracting the image data from the streaming media file and generating target video data corresponding to the image data comprises:
and extracting image data from the streaming media file according to the preset video frame rate, and generating target video data corresponding to the image data.
6. A game live method in a live client according to any one of claims 1 to 3, wherein encoding the live image comprises:
and zooming the live broadcast image, and encoding the zoomed live broadcast image.
7. A game live method in a live client according to claim 2 or 3, wherein before generating the streaming media file, the game live method in the live client further comprises:
and adding protection information to the encoded live image.
8. The live game playing method in the live client according to claim 2, wherein encoding the live image comprises:
and zooming the live broadcast image, and encoding and adding protection information to the zoomed live broadcast image.
9. A game live method in a live client according to claim 2 or 3, wherein the game live method in the live client further comprises:
determining available storage space of a file library for storing the streaming media file;
and if the streaming media file is larger than the available storage space, clearing the file library, recoding the live broadcast image, and generating the streaming media file to be stored in the file library.
10. A game live method in a live client according to any one of claims 1 to 3, wherein the game live method in the live client further comprises:
determining the encoding time length for encoding the live broadcast image;
and if the encoding duration is greater than a preset threshold value, re-encoding the live broadcast image.
11. A game live broadcast device in a live broadcast client, wherein the live broadcast client runs on an electronic terminal, and the electronic terminal comprises a game client, and the game live broadcast device is characterized by comprising:
the image acquisition module is used for acquiring a live image of the game through an image rendering thread of the game client;
the audio acquisition module is used for acquiring the live audio of the game;
and the data coding module is used for coding the live broadcast image and the live broadcast audio and pushing the coded live broadcast image and the live broadcast audio to a live broadcast server.
12. A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements a method of live gaming in a live client as claimed in any one of claims 1 to 10.
13. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement a method of live gaming in a live client as claimed in any one of claims 1 to 10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911071870.7A CN110784757A (en) | 2019-11-05 | 2019-11-05 | Game live broadcast method and device in live broadcast client, medium and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201911071870.7A CN110784757A (en) | 2019-11-05 | 2019-11-05 | Game live broadcast method and device in live broadcast client, medium and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN110784757A true CN110784757A (en) | 2020-02-11 |
Family
ID=69389136
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201911071870.7A Pending CN110784757A (en) | 2019-11-05 | 2019-11-05 | Game live broadcast method and device in live broadcast client, medium and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110784757A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111330281A (en) * | 2020-03-05 | 2020-06-26 | 网易(杭州)网络有限公司 | Game data processing method, game data processing device, medium, and electronic device |
CN113056030A (en) * | 2021-02-01 | 2021-06-29 | 广州朗国电子科技有限公司 | Wireless screen projection module and screen projection method of all-in-one machine |
CN113596502A (en) * | 2021-08-03 | 2021-11-02 | 广州繁星互娱信息科技有限公司 | Live broadcast room sound effect adjusting method and device, electronic equipment and medium |
CN114390300A (en) * | 2020-10-20 | 2022-04-22 | 腾讯科技(深圳)有限公司 | Live broadcast control method and device, electronic equipment and computer storage medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8381259B1 (en) * | 2012-01-05 | 2013-02-19 | Vinod Khosla | Authentication and synchronous interaction between a secondary device and a multi-perspective audiovisual data stream broadcast on a primary device |
US20140259084A1 (en) * | 2009-09-22 | 2014-09-11 | Caption Colorado Llc | Caption and/or Metadata Synchronization for Replay of Previously or Simultaneously Recorded Live Programs |
CN105407361A (en) * | 2015-11-09 | 2016-03-16 | 广州华多网络科技有限公司 | Audio and video live broadcast data processing method and device |
CN106210753A (en) * | 2016-05-24 | 2016-12-07 | 北京畅游天下网络技术有限公司 | A kind of net cast method and device |
CN106303329A (en) * | 2016-08-11 | 2017-01-04 | 广州爱九游信息技术有限公司 | Record screen live broadcasting method and device, mobile device and live broadcast system |
CN109168014A (en) * | 2018-09-26 | 2019-01-08 | 广州虎牙信息科技有限公司 | A kind of live broadcasting method, device, equipment and storage medium |
-
2019
- 2019-11-05 CN CN201911071870.7A patent/CN110784757A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140259084A1 (en) * | 2009-09-22 | 2014-09-11 | Caption Colorado Llc | Caption and/or Metadata Synchronization for Replay of Previously or Simultaneously Recorded Live Programs |
US8381259B1 (en) * | 2012-01-05 | 2013-02-19 | Vinod Khosla | Authentication and synchronous interaction between a secondary device and a multi-perspective audiovisual data stream broadcast on a primary device |
CN105407361A (en) * | 2015-11-09 | 2016-03-16 | 广州华多网络科技有限公司 | Audio and video live broadcast data processing method and device |
CN106210753A (en) * | 2016-05-24 | 2016-12-07 | 北京畅游天下网络技术有限公司 | A kind of net cast method and device |
CN106303329A (en) * | 2016-08-11 | 2017-01-04 | 广州爱九游信息技术有限公司 | Record screen live broadcasting method and device, mobile device and live broadcast system |
CN109168014A (en) * | 2018-09-26 | 2019-01-08 | 广州虎牙信息科技有限公司 | A kind of live broadcasting method, device, equipment and storage medium |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111330281A (en) * | 2020-03-05 | 2020-06-26 | 网易(杭州)网络有限公司 | Game data processing method, game data processing device, medium, and electronic device |
CN111330281B (en) * | 2020-03-05 | 2023-03-28 | 网易(杭州)网络有限公司 | Game data processing method, game data processing device, medium, and electronic device |
CN114390300A (en) * | 2020-10-20 | 2022-04-22 | 腾讯科技(深圳)有限公司 | Live broadcast control method and device, electronic equipment and computer storage medium |
CN114390300B (en) * | 2020-10-20 | 2024-04-26 | 腾讯科技(深圳)有限公司 | Live broadcast control method and device, electronic equipment and computer storage medium |
CN113056030A (en) * | 2021-02-01 | 2021-06-29 | 广州朗国电子科技有限公司 | Wireless screen projection module and screen projection method of all-in-one machine |
CN113596502A (en) * | 2021-08-03 | 2021-11-02 | 广州繁星互娱信息科技有限公司 | Live broadcast room sound effect adjusting method and device, electronic equipment and medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110784757A (en) | Game live broadcast method and device in live broadcast client, medium and electronic equipment | |
CN112291627B (en) | Video editing method and device, mobile terminal and storage medium | |
US10229651B2 (en) | Variable refresh rate video capture and playback | |
WO2016150317A1 (en) | Method, apparatus and system for synthesizing live video | |
US9363542B2 (en) | Techniques to provide an enhanced video replay | |
CN109168014B (en) | Live broadcast method, device, equipment and storage medium | |
KR101387212B1 (en) | System and method for generating and playing three dimensional image files based on two dimensional image media standards | |
CN110324643B (en) | Video recording method and system | |
JP2016527791A (en) | Image processing method and apparatus | |
CN112272327B (en) | Data processing method, device, storage medium and equipment | |
JP2018521550A (en) | Method, client and computer storage medium for playing video | |
KR101388265B1 (en) | System and method for generating and playing three dimensional image files based on two dimensional image media standards | |
CN113535063A (en) | Live broadcast page switching method, video page switching method, electronic device and storage medium | |
CN114697668A (en) | Encoding and decoding method of point cloud media and related product | |
CN111918082A (en) | Cloud game live broadcasting method and device, storage medium and equipment | |
JP2024506623A (en) | Point cloud encoding/decoding method, device, computer readable medium, and electronic device | |
CN111918074A (en) | Live video fault early warning method and related equipment | |
KR101085718B1 (en) | System and method for offering augmented reality using server-side distributed image processing | |
CN111093091A (en) | Video processing method, server and system | |
CN112954374B (en) | Video data processing method and device, electronic equipment and storage medium | |
CN113395531B (en) | Play switching method and device, electronic equipment and computer readable storage medium | |
CN110798700B (en) | Video processing method, video processing device, storage medium and electronic equipment | |
CN112235600B (en) | Method, device and system for processing video data and video service request | |
JP2024508865A (en) | Point cloud encoding/decoding method, device, and electronic equipment | |
JP7376235B2 (en) | Video distribution system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200211 |