CN108391179A - live data processing method, device, server, terminal and storage medium - Google Patents

live data processing method, device, server, terminal and storage medium Download PDF

Info

Publication number
CN108391179A
CN108391179A CN201810388944.9A CN201810388944A CN108391179A CN 108391179 A CN108391179 A CN 108391179A CN 201810388944 A CN201810388944 A CN 201810388944A CN 108391179 A CN108391179 A CN 108391179A
Authority
CN
China
Prior art keywords
target
target video
video
interactive information
video fragment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810388944.9A
Other languages
Chinese (zh)
Other versions
CN108391179B (en
Inventor
左洪涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201810388944.9A priority Critical patent/CN108391179B/en
Publication of CN108391179A publication Critical patent/CN108391179A/en
Application granted granted Critical
Publication of CN108391179B publication Critical patent/CN108391179B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4758—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for providing answers, e.g. voting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream

Abstract

The embodiment of the invention discloses a kind of live data processing method, device, server, terminal and storage medium, wherein method includes：Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；Video stream data is handled to obtain the target video fragment of live video, and obtains the address of target video fragment；Target interactive information associated with target video fragment is chosen from least one interactive information；The index of target video fragment is established in the index file of live video, the index of target video fragment includes address and the target interactive information of target video fragment；When the video request that the spectator client for receiving live video is sent, index file is sent to spectator client.The real-time, interactive between main broadcaster and spectators may be implemented in the embodiment of the present invention, simplifies live streaming interaction flow, improves the interactive efficiency during live streaming, reduce the processing load of client and server, promotes practicability and user experience.

Description

Live data processing method, device, server, terminal and storage medium
Technical field
The present invention relates to Internet technical fields, and in particular to technical field of data processing more particularly to a kind of live streaming number According to processing method, a kind of live data processing unit, a kind of server being used to be broadcast live data processing, one kind for carrying out The terminal of live data processing and a kind of computer storage media.
Background technology
Streaming media video live streaming refers to being sent out using stream media technology while acquiring media information (such as audio/video information) The media information that cloth and transmission are acquiring enables the client to play the media information acquired by internet.Currently, The interaction of main broadcaster and spectators can be achieved in live video in playing process, and principle is mainly：Main broadcaster's user end to server is sent out Mutual information, server storage interactive information are delivered, spectator client obtains currently playing with some cycles (such as 2 seconds, 3 seconds) The serial number of the TS fragments is transferred to server request and handed over by the serial number of TS (Transport Stream, transfer stream file) fragment Mutual information；Server judges whether the TS fragments have associated interactive information after receiving the request；If so, then to spectators visitor Family end returns to interactive information and is shown.In interaction technique in above-mentioned existing streaming media video live streaming, need to regard in live streaming Real time request and transmission interactive information in the playing process of frequency, this can lead to interactive delay, reduce the experience of user；Moreover, seeing Many client timings are sent to server asks, processing load that is cumbersome and increasing spectator client and server, If the unreasonable acquisition missing that can also result in interactive information of the setting of timing cycle, less efficient and practicability is poor, uses Family experience is poor.
Invention content
An embodiment of the present invention provides a kind of live data processing method, device, server, terminal and storage mediums, can To realize the real-time, interactive between main broadcaster and spectators, the processing load of client and server is reduced, promotes practicability and user Experience.
An embodiment of the present invention provides a kind of live data processing method, the live data processing method includes：
Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
The video stream data is handled to obtain the target video fragment of the live video, and obtains the target The address of video slicing；
Target interactive information associated with the target video fragment is chosen from least one interactive information；
The index of the target video fragment, the target video fragment are established in the index file of the live video Index include the address of the target video fragment and the target interactive information；
When the video request that the spectator client for receiving the live video is sent, the index file is sent to The spectator client, makes the spectator client include according to the index of target video fragment described in the index file The address of the target video fragment obtains and plays the target video fragment, and in broadcasting circle of the target video fragment Face shows the target interactive information.
Another embodiment of the present invention provides a kind of live data processing method, and the live data processing method includes：
The spectator client of live video sends video request to server；
The index file for the live video that the server returns is received, the index file includes the server The index of the index established by the target video fragment of the live video, the target video fragment includes that the target regards The address of frequency division piece and target interactive information associated with the target video fragment；Wherein, the target video fragment by The video stream data that the server sends main broadcaster's client of the live video is handled to obtain；The target video The address of fragment is acquired by the server；The target interactive information is sent out by the server from main broadcaster's client It chooses and obtains at least one interactive information sent；
The ground of the target video fragment is read in the index for the target video fragment for including from the index file Location and the target interactive information；
It is obtained according to the address of the target video fragment and plays the target video fragment, and in the target video The broadcast interface of fragment shows the target interactive information.
The embodiment of the present invention additionally provides a kind of live data processing unit, is applied to server, live data processing Device includes：
Receiving unit, the video stream data and at least one interaction letter that main broadcaster's client for receiving live video is sent Breath；
Processing unit obtains the target video fragment of the live video for being handled the video stream data, And obtain the address of the target video fragment；
Selection unit, for choosing mesh associated with the target video fragment from least one interactive information Mark interactive information；
Unit is established, the index for establishing the target video fragment in the index file of the live video, institute The index for stating target video fragment includes the address of the target video fragment and the target interactive information；
Output unit will be described when video request for being sent when the spectator client for receiving the live video Index file is sent to the spectator client, makes the spectator client according to target video described in the index file point The address for the target video fragment that the index of piece includes obtains and plays the target video fragment, and is regarded in the target The broadcast interface of frequency division piece shows the target interactive information.
Another embodiment of the present invention provides a kind of live data processing unit, is applied to the spectators client of live video End, the live data processing unit include：
Request unit, for sending video request to server；
Receiving unit, the index file for receiving the live video that the server returns, the index file Including the index that the server is established by the target video fragment of the live video, the index of the target video fragment Address including the target video fragment and target interactive information associated with the target video fragment；Wherein, described The video stream data that target video fragment sends main broadcaster's client of the live video by the server handle It arrives；The address of the target video fragment is acquired by the server；The target interactive information by the server from It chooses and obtains at least one interactive information that main broadcaster's client is sent；
Reading unit reads the target in the index of the target video fragment for including from the index file The address of video slicing and the target interactive information；
Processing unit, for the target video fragment to be obtained and played according to the address of the target video fragment, and The target interactive information is shown in the broadcast interface of the target video fragment.
The embodiment of the present invention additionally provides a kind of server for being broadcast live data processing, which includes：
Processor is adapted for carrying out one or one or more instruction；And
Computer storage media, the computer storage media be stored with one or one or more instruction, described one or One or more instruction is suitable for being loaded by the processor and executing following steps：
Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
The video stream data is handled to obtain the target video fragment of the live video, and obtains the target The address of video slicing；
Target interactive information associated with the target video fragment is chosen from least one interactive information；
The index of the target video fragment, the target video fragment are established in the index file of the live video Index include the address of the target video fragment and the target interactive information；
When the video request that the spectator client for receiving the live video is sent, the index file is sent to The spectator client, makes the spectator client include according to the index of target video fragment described in the index file The address of the target video fragment obtains and plays the target video fragment, and in broadcasting circle of the target video fragment Face shows the target interactive information.
The embodiment of the present invention additionally provides a kind of terminal for being broadcast live data processing, and the terminal operating has live streaming The spectator client of video, the terminal include input equipment and output equipment, further include：
Processor is adapted for carrying out one or one or more instruction；And
Computer storage media, the computer storage media be stored with one or one or more instruction, described one or One or more instruction is suitable for being loaded by the processor and executing following steps：
The spectator client of live video sends video request to server；
The index file for the live video that the server returns is received, the index file includes the server The index of the index established by the target video fragment of the live video, the target video fragment includes that the target regards The address of frequency division piece and target interactive information associated with the target video fragment；Wherein, the target video fragment by The video stream data that the server sends main broadcaster's client of the live video is handled to obtain；The target video The address of fragment is acquired by the server；The target interactive information is sent out by the server from main broadcaster's client It chooses and obtains at least one interactive information sent；
The ground of the target video fragment is read in the index for the target video fragment for including from the index file Location and the target interactive information；
It is obtained according to the address of the target video fragment and plays the target video fragment, and in the target video The broadcast interface of fragment shows the target interactive information.
The embodiment of the present invention additionally provides a kind of computer storage media, the computer storage media be stored with one or One or more first instruction, described one or one or more first instruction be suitable for loaded by processor and execute following steps：
Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
The video stream data is handled to obtain the target video fragment of the live video, and obtains the target The address of video slicing；
Target interactive information associated with the target video fragment is chosen from least one interactive information；
The index of the target video fragment, the target video fragment are established in the index file of the live video Index include the address of the target video fragment and the target interactive information；
When the video request that the spectator client for receiving the live video is sent, the index file is sent to The spectator client, makes the spectator client include according to the index of target video fragment described in the index file The address of the target video fragment obtains and plays the target video fragment, and in broadcasting circle of the target video fragment Face shows the target interactive information.
Another embodiment of the present invention provides a kind of computer storage media, and the computer storage media is stored with one Or one or more second instruction, described one or one or more second instruction be suitable for loaded by processor and execute following steps：
The spectator client of live video sends video request to server；
The index file for the live video that the server returns is received, the index file includes the server The index of the index established by the target video fragment of the live video, the target video fragment includes that the target regards The address of frequency division piece and target interactive information associated with the target video fragment；Wherein, the target video fragment by The video stream data that the server sends main broadcaster's client of the live video is handled to obtain；The target video The address of fragment is acquired by the server；The target interactive information is sent out by the server from main broadcaster's client It chooses and obtains at least one interactive information sent；
The ground of the target video fragment is read in the index for the target video fragment for including from the index file Location and the target interactive information；
It is obtained according to the address of the target video fragment and plays the target video fragment, and in the target video The broadcast interface of fragment shows the target interactive information.
The video stream data and at least one friendship that the embodiment of the present invention is sent in the main broadcaster's client for receiving live video After mutual information, video stream data and at least one interactive information can be handled, obtain target video fragment, target The address of video slicing and target interactive information associated with target video fragment.And in the index file of live video The index of target video fragment is established, the index of the target video fragment includes address and the target interaction letter of target video fragment Breath；When the video request that the spectator client for receiving the live video is sent, the index file is sent to described Spectator client；Since the index of target video fragment includes target interactive information, and the index of target video fragment is to include In the index file of live video, then, the index for the target video fragment that spectator client can include from index file In directly read target interactive information, avoid constantly to server timing request in the playing process of live video, thus It avoids and frequently interacts caused resource consumption between client and server；Spectator client can be in target video fragment Broadcast interface display target interactive information, to realize the real-time, interactive between main broadcaster and spectators, live streaming interactive efficiency is high, Practicability is high and user experience is good.
Description of the drawings
Technical solution in order to illustrate the embodiments of the present invention more clearly, below will be to needed in embodiment description Attached drawing is briefly described, it should be apparent that, drawings in the following description are some embodiments of the invention, general for this field For logical technical staff, without creative efforts, other drawings may also be obtained based on these drawings.
Fig. 1 is a kind of configuration diagram of live broadcast system provided in an embodiment of the present invention；
Fig. 2 is a kind of internal structure schematic diagram of live broadcast system provided in an embodiment of the present invention；
Fig. 3 is a kind of flow diagram of live data processing method provided in an embodiment of the present invention；
Fig. 4 is a kind of flow diagram for live data processing method that another embodiment of the present invention provides；
Fig. 5 a are a kind of broadcast interface schematic diagrames of spectator client provided in an embodiment of the present invention；
Fig. 5 b are the broadcast interface schematic diagrames of another spectator client provided in an embodiment of the present invention；
Fig. 5 c are the broadcast interface schematic diagrames of another spectator client provided in an embodiment of the present invention；
Fig. 6 a are the broadcast interface schematic diagrames of another spectator client provided in an embodiment of the present invention；
Fig. 6 b are the broadcast interface schematic diagrames of another spectator client provided in an embodiment of the present invention；
Fig. 7 a are a kind of application scenarios schematic diagrams of spectator client provided in an embodiment of the present invention；
Fig. 7 b are the broadcast interface schematic diagrames of another spectator client provided in an embodiment of the present invention；
Fig. 8 is a kind of interaction scenarios schematic diagram of live data processing method provided in an embodiment of the present invention；
Fig. 9 is a kind of structural schematic diagram of live data processing unit provided in an embodiment of the present invention；
Figure 10 is a kind of structural schematic diagram for live data processing unit that another embodiment of the present invention provides；
Figure 11 is a kind of structural schematic diagram of server provided in an embodiment of the present invention；
Figure 12 is a kind of structural schematic diagram of terminal provided in an embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation describes.
Net cast refers to spectators can watch that the ongoing live audio and video of main broadcaster's client are real by internet Condition, such as competitive sports, teaching etc..Streaming Media can be described as streaming video again, refer to it is a kind of using stream transmission by the way of The media formats played on internet.Net cast based on Streaming Media can be referred to as streaming media video live streaming, the stream matchmaker Volumetric video live streaming refers to that the media acquired are issued and transmitted using stream media technology and are believed while acquiring audio/video information Breath enables spectator client to play the media information acquired by internet.The stream media technology is passed using streaming Defeated mode transmits multimedia file on the internet, and the multimedia file may include audio-video document, then continuous Audio and video are issued after compression processing and are stored in web medium server, are played back while downloading for spectator client, It all downloads and completes without waiting for whole multimedia files.
It is realized in the live broadcast system that streaming media video live streaming can be shown in Fig. 1.As shown in Figure 1, live broadcast system may include Server 11, main broadcaster's client 12 and at least one spectator client 13.Wherein, main broadcaster's client 12 refers to publication live video Client used by a user.Spectator client 13 refers to request viewing and plays the visitor that the user of the direct video uses Family end.Main broadcaster's client 13 and spectator client 13 may each be and run in terminal, terminal herein can be mobile phone, The portable devices and desktop computer etc. such as laptop computer or tablet computer.Server 11 can be for carrying Can be data processing server, web server, the corresponding application clothes of net cast APP for the service equipment of direct broadcast service Business device etc.；Server can be an independent service equipment, can also be the cluster collectively formed by multiple service equipments Equipment.
In the live broadcast system, the implementation process of streaming media video live streaming is substantially：1. main broadcaster's client of live video Video stream data is sent in real time to server；2. server acquires the video stream data of main broadcaster's client transmission in real time, and to institute It states video stream data and carries out coded treatment generation video slicing；Video slicing herein can refer to TS fragments；3. server storage The TS fragments generated, and the attribute informations such as address, length, serial number for obtaining the TS fragments, and given birth to according to the attribute information At the index of the TS fragments, then the index of the TS fragments is added in the index file of the live video；4. live streaming regards The spectator client of frequency sends video request to server, and the index file of the live video is returned to the sight by server Many clients；5. spectator client after receiving the index file, can parse to obtain the index file The index of TS fragments is taken, and finds the address of the TS fragments according to the index, the TS fragments is then downloaded and plays out.It needs Illustrate, after video starts live streaming, the video stream data that server receives every time is one section of video in the live video Flow data, the video stream data can be by real-time codings at one or more TS fragments, and the quantity of the TS fragments generated is by video The playing duration of flow data and the playing duration of each TS fragments determine；For example, the playing duration of TS fragments is 2 seconds, video flowing The playing duration of data is 4 seconds, then server can handle the video stream data to obtain 2 TS fragments；For another example, TS fragments Playing duration is 2 seconds, and the playing duration of video stream data is 2 seconds, then server can handle the video stream data to obtain 1 TS fragments.The index of each TS fragments can also be generated and is added in index file in real time in real time, and index file is real time updated To spectator client to realize the continuous real-time broadcasting of live video in spectator client.For ease of description, it removes and especially says Bright outer, it can be the video fluxion that server sends main broadcaster's client that target video fragment is obtained mentioned by the embodiment of the present invention According to any one TS fragment in the TS fragments handled.
During streaming media video is broadcast live, main broadcaster can also send interactive information by main broadcaster's user end to server, Server receives the interactive information and is sent to spectator client, to realize the interaction between main broadcaster and spectators.In order to realize Real-time, interactive between main broadcaster and spectators reduces interaction time delay, process is broadcast live for above-mentioned streaming media video, the present invention is implemented Example has carried out following improvement：1. main broadcaster's user end to server of live video sends video stream data and at least one friendship in real time Mutual information；2. server acquires the video stream data and at least one interactive information that main broadcaster's client is sent in real time, and right The video stream data is handled to obtain the target video fragment of the live video, and obtains the ground of the target video fragment Location；3. choosing target interactive information associated with target video fragment from least one interactive information；4. in live video Index file in establish the index of target video fragment, the index of the target video fragment includes the address of target video fragment And target interactive information；5. the spectator client of live video sends video request to server, server regards the live streaming The index file of frequency returns to the spectator client；6. spectator client, can be right after receiving the index file The index file is parsed to obtain the index of target video fragment, which not only includes the ground of target video fragment Location, also include the associated target interactive information of target video fragment, then spectator client can download target video fragment into Row plays, and the display target interactive information in broadcast interface.Live data processing method in the embodiment of the present invention at least has It has the following advantages：(1) server can be risen target video fragment and target interactive information one by way of index file Spectator client is given, is carried out searching the operations such as target interactive information according to the serial number of target video fragment without executing, it can be with Target interactive information is sent to spectator client in time, interactive promptness is improved, to improve the experience of user；(2) by In the index of target video fragment include the associated target interactive information of target video fragment, therefore spectator client according to Index file would know that whether target video file has associated interactive information, and can directly acquire target interaction letter Breath need not periodically send the request of interactive information to server, save resource.
In the embodiment of the present invention, live streaming agreement is followed between client and server during streaming media video live streaming, it should Live streaming agreement may include but be not limited to：RTMP (Real Time Messaging Protocol, real-time messages transport protocol) is assisted View, HLS (HTTP Live Streaming) agreement etc..Unless otherwise indicated, subsequent embodiment of the present invention is with HLS protocol For, HLS protocol is a kind of streaming media solutions, includes (TS points of index file (m3u8 files), media slicing file Piece) and encryption string file (key).Wherein, m3u8 files refer to the m3u files of UTF-8 coded formats, be one have recorded it is more The text-only file of the index of a TS fragments.Spectator client is not direct broadcasting m3u8 texts in playing audio-video document Part, but m3u8 files are parsed, the index of each TS fragments is obtained, and corresponding TS fragments are found according to the index Address plays out.TS fragments are the minimum broadcasting content lists under video slicing and HLS protocol under the index of m3u8 files Member, each TS fragments are one section of live video content, and all TS fragments in m3u8 files form complete live video.
Please also refer to the internal structure of live broadcast system shown in Fig. 2.As shown in Fig. 2, main broadcaster's client 12 includes mainly 122 liang of big function modules of network module 121 and protocol process module.Spectator client 13 includes mainly：Network module 131, association View parsing module 132 decodes and renders module 133 and interactive information logic processing module 134.Server may include following Function module：Net cast acquisition module 111, coding module 112, storing address generating module 113, interactive information management module 114, content distribution module 115.The implementation principle of each function module is specific as follows：
Main broadcaster's client 12 is handled video stream data and interactive information by protocol process module 122, and is passed through Network module 121 is sent to server.
The video that server 11 passes through the main broadcaster's client transmission of acquisition live video in real time of net cast acquisition module 111 Flow data and at least one interactive information.It, can be by least one interactive information after collecting at least one interactive information Store interactive information management module 114；After collecting video stream data, collected video stream data can be transmitted To coding module 112, coded treatment is carried out according to collected video stream data by coding module 112 and generates video slicing.So Video slicing is distributed by content distribution module 115 afterwards, in storage to corresponding storage system.Storing address generating module 113, when generating the index of target video fragment, can obtain the address of target video fragment, and to interactive information management module 114 ask and obtain target interactive information associated with target video fragment.Mesh is established in the index file of live video The index of video slicing is marked, the index of target video fragment includes address and the target interactive information of target video fragment.
Spectator client 13 can send video request to server by network module 131 and receive what server returned The index file of live video.Dissection process is carried out to index file by protocol resolution module 132, is regarded with target if getting The associated target interactive information of frequency division piece, then be transmitted to interactive information logic processing module 134 by target interactive information.Interaction Infologic processing module 134, can will be at the Context resolution in target interactive information after receiving target interactive information Reason, and display is carried out to content either other logical process (for example displaying topic, answer or announce current guess and are in progress Deng).If getting target video fragment, target video fragment is stored in buffering area, and by decoding and rendering module 133 to slow The buffered data rushed in area is decoded and renders, and is played out in the broadcast interface of spectator client.
Description based on above-described embodiment, the embodiment of the present invention propose a kind of live data processing method, the live streaming number It can be executed by server shown in fig. 1 or fig. 2 according to processing method.Fig. 3 is referred to, which may include following Step S101-S105：
S101 receives video stream data and at least one interactive information that main broadcaster's client of live video is sent.
Main broadcaster's client of the live video refers to the client issued corresponding to the user of the live video.Regarding During frequency is broadcast live, server can acquire the video stream data and at least one interaction that main broadcaster's client of live video is sent Information.Interactive information herein may include following at least one：Text message, voice messaging, image information etc..
S102 handles video stream data to obtain the target video fragment of live video, and obtains the target video The address of fragment.
Server can carry out the video stream data coded treatment generation target and regard after obtaining video stream data Frequency division piece (target TS fragments).In one embodiment, due to during live video, to the more demanding of real-time, Therefore when generating target video fragment, the playing duration that can control the target video fragment is less than the first preset length threshold Value, first pre-set length threshold can be determined according to actual business demand.Such as it can be by the target video fragment Playing duration control in 1-2 seconds, real-time is improved with this.
After coding is completed, server can obtain one or more TS fragments.Server can be a or more by this In a TS fragments storage to corresponding storage system.In one embodiment, server can be by the one or more TS fragments It stores in content distribution network system, such as CDN (Content DeliveryNetwork, content distributing network).By described The one or more TS fragments are distributed to closest in the storage server of spectator client by CDN so that spectator client can To get TS fragments with faster speed.Therefore, server can store the target video fragment to content distributing network In system；The address of the target video fragment is obtained from the content distribution network system.
S103 chooses target interactive information associated with target video fragment from least one interactive information.
Since server can obtain one or more TS fragments, server after carrying out coded treatment to video stream data Received interactive information may be one or more, and therefore, server needs are chosen from least one interactive information Target interactive information associated with target video fragment.In one embodiment, server can be according to target video fragment Including the receiving time of receiving time and at least one interactive information of video stream data determine.Specifically, server can To obtain the receiving time for the video stream data that target video fragment includes；Obtain the receiving time of at least one interactive information； According to the receiving time of at least one interactive information, chooses from least one interactive information and regarded with what target video fragment included The interactive information that the receiving time of frequency flow data matches；Selected interactive information is determined as related to target video fragment The target interactive information of connection.For example, server is 11:10:A length of 6 seconds video stream datas, TS fragments when being had received when 06 Playing duration be 2 seconds, then server can be obtained according to the video stream data TS1, TS2 and TS3 totally 3 TS fragments and The receiving time for the video stream data that this 3 TS fragments are included, as shown in table 1.
Table 1
Video slicing Duration Including video stream data receiving time
TS1 2 seconds 11:10:06-11:10:08
TS2 2 seconds 11:10:08-11:10:10
TS3 2 seconds 11:10:10-11:10:12
If TS1 is target video fragment, the receiving time for getting interactive information 1 is 11:10:07, interactive information 2 connects It is 11 between time receiving:10:09, then it can determine the video fluxion that the receiving time of interactive information 1 includes with target video fragment TS1 According to receiving time match, interacted so as to which interactive information 1 is determined as associated with target video fragment TS1 target Information.
S104 establishes the index of target video fragment in the index file of live video, the rope of the target video fragment Draw the address including target video fragment and target interactive information.
Server is in the address and target interactive information associated with target TS fragments for having got target TS fragments After, the index of target video fragment can be established in the index file of live video.It in one embodiment, can be in institute It is the target TS fragments distribution index item to state in the index file of live video, and the index entry includes data field and extension Field；The address of the target TS fragments is stored into the data field, and the target interactive information is stored to institute It states in extended field.
In one embodiment, server can be stored the target interactive information to described in the form of program code In extended field, such as extended field name can be " #EXT-QQLIVE-QUESTION ", and the content of target interactive information is “question：" in 1. professional basketball games, scoring a goal can obtain rather " option：“1：1 point " " 2：2 points " " 3：3 points " ", It can be as follows by the program code in the content write-in extended field of the target interactive information：
#EXT-QQLIVE-QUESTION:{“question”:" in 1. professional basketball games, scoring a goal, it is several to obtain Point " " option "=[{ { " 1 "：" 1 point " } { " 2 "：" 2 points " } { " 3 points "：" 3 points " } }] }
In another embodiment, server can store the target interactive information into a storage address, then It will be in storage address storage to the extended field.If the target interactive information includes in image and audio and video etc. Hold, server is before storing the target interactive information into extended field, it may be necessary to described image or described Audio and video carry out transcoding processing, reduce the formation efficiency of index file.By by the target interactive information storage to storage Address, so that spectator client obtains the interactive information by the storage address, without carrying out transcoding processing, so as to Improve the formation efficiency of index file.
It should be noted that if the target interactive information that target TS fragments are not associated, then the rope of target TS fragments Draw include target TS fragments address, i.e. index entry can only include data field, and the address storage of target TS fragments is arrived In the data field.For example, when server is in the index for establishing TS1, target interactive information associated with TS1 has been got (interactive information 1), then the index of TS1 includes address and the interactive information 1 of TS1.When index of the server in generation TS2, do not have Target interactive information associated with TS2 is got, then the index of TS2 can only include the address of TS2.
In one embodiment, the index of the target TS fragments further includes the playing duration of target TS fragments, described to broadcast It puts duration and is less than the first pre-set length threshold；The first pre-set length threshold herein can be set according to actual needs, example Such as：First pre-set length threshold is 1 second, 2 seconds etc..
Index file is sent to sight by S105 when the video request that the spectator client for receiving live video is sent Many clients, the mesh for making the spectator client include according to the index of target video fragment described in the index file The address of mark video slicing obtains and plays the target video fragment, and is shown in the broadcast interface of the target video fragment The target interactive information.
In one embodiment, video request carries the mark of live video, and mark herein can be live video Download address, title of live video etc..Server, can basis after the video request for receiving spectator client transmission Identification information entrained by video request obtains the index file of the live video.Each live video can correspond to a mark Know information, the identification information is spectator client according to spectators' determination to the selection instruction of live video.
The video stream data and at least one friendship that the embodiment of the present invention is sent in the main broadcaster's client for receiving live video After mutual information, video stream data and at least one interactive information can be handled, obtain target video fragment, target The address of video slicing and target interactive information associated with target video fragment.And in the index file of live video The index of target video fragment is established, the index of the target video fragment includes address and the target interaction letter of target video fragment Breath；When the video request that the spectator client for receiving the live video is sent, the index file is sent to described Spectator client；Since the index of target video fragment includes target interactive information, and the index of target video fragment is to include In the index file of live video, then, the index for the target video fragment that spectator client can include from index file In directly read target interactive information, avoid constantly to server timing request in the playing process of live video, thus It avoids and frequently interacts caused resource consumption between client and server；Spectator client can be in target video fragment Broadcast interface display target interactive information, to realize the real-time, interactive between main broadcaster and spectators, live streaming interactive efficiency is high, Practicability is high and user experience is good.
Description based on above-described embodiment, another embodiment of the present invention additionally provide a kind of stream of live data processing method Journey schematic diagram, the live data processing method can be executed by spectator client shown in fig. 1 or fig. 2.Fig. 4 is referred to, this is straight Multicast data processing method may include following steps S201-S204：
The spectator client of S201, live video send video request to server.
The spectator client of the live video refers to the client watched corresponding to the user of the live video, described The quantity of spectator client can be with more than one.The spectator client of live video can be after getting triggering command, to clothes Device transmission video request be engaged in obtain the index file of the live video.In one embodiment, the triggering command can be with It is to input the operation of live video network address in the browser of spectator client according to the spectators that detect to be generated, it can also It is the behaviour that some live video APP (Application, application program) in spectator client is opened according to the spectators detected What work was generated.
Since server can generate multiple index files, an index file corresponds to a live video, therefore, one In a embodiment, the video request can carry the identification information of the live video, and the identification information can be used for referring to Show that server determines the live video；For example, wanting to see live in the period of video in spectators, there are three live streamings to regard in total Frequently, it is competitive sports, live streaming answer and live broadcast teaching respectively, and the live video of spectators' selection is competitive sports, then seeing Many clients can carry the mark letter of the competitive sports when sending video request to server in the video request Breath, to indicate that the index file of competitive sports is returned to spectator client by server.
S202, receives the index file for the live video that server returns, which includes that server is that live streaming regards The index of the index that the target video fragment of frequency is established, target video fragment includes address and and the target of target video fragment The associated target interactive information of video slicing.
Wherein, the video stream data that target video fragment sends main broadcaster's client of live video by server carries out Reason obtains；The address of target video fragment is acquired by server；Target interactive information is sent out by server from main broadcaster's client It chooses and obtains at least one interactive information sent.
Spectator client can carry out index file after receiving the index file of live video of server return Dissection process can execute step S203 to obtain the index of target TS fragments after the index for getting target TS fragments.
S203 reads the ground of the target video fragment from the index for the target video fragment that the index file includes Location and target interactive information.
S204 obtains according to the address of target video fragment and plays target video fragment, and in target video fragment Broadcast interface display target interactive information.
In one embodiment, index file includes the index entry of target TS fragments, and the index of target TS fragments corresponds to This index entry, the index entry include data field and extended field；The address of the target TS fragments is stored in the data In field, the target interactive information is stored in the extended field.Corresponding, spectator client is executing step S203 When, it can be to search the index entry of target TS fragments in indexed file；The mesh is obtained from the data field of the index entry The address of TS fragments is marked, and obtains the target interactive information from the extended field of the index entry.Pass through step S203 Behind the address for getting target TS fragments, it can be obtained from content distribution network system according to the address of the target TS fragments The target TS fragments, and play the target TS fragments；When playing target TS fragments, target TS fragments can be added It is loaded onto buffering area, is decoded by buffering area and is rendered, the target video fragment that play buffer is loaded.The buffering area Length is less than the second pre-set length threshold.When the length of buffering area can determine delay of the player when playing live video It is long, if such as the length of buffering area is 1.75 seconds, illustrate player at most delay 1.75 seconds when playing live video.It is described Second pre-set length threshold can be determined according to practical business demand.
In one embodiment, if what spectator client obtained from the extended field of the index entry is network Location (i.e. the storage address of target interactive information) then obtains interactive information according to the network address, and at the target TS points The broadcast interface of piece shows the target interactive information.
In one embodiment, spectator client, can in the broadcast interface display target interactive information of target TS fragments The broadcast interface is divided into the first play area and the second play area；It is put in first play area relaying continued broadcasting The target TS fragments show the target interactive information, as shown in Figure 5 a in second play area.Another is implemented In example, interactive window can be popped up in the broadcast interface；It continues to play the target TS fragments in the broadcast interface, The target interactive information is shown in the interactive window, as shown in Figure 5 b.It, can be in the broadcasting in further embodiment Interface shows the target interactive information in the form of message informing column, as shown in Figure 5 c.It should be noted that listed above The broadcast interface display target interactive information in target TS fragments mode only illustrate, it is not exhaustive.
In one embodiment, the interactive information may include following at least one：Problem and option, answer, guess Topic, guess result.Spectator client can also collect audience interaction information of the live video in playing process, and right The audience interaction information carries out business processing；The audience interaction information may include following at least one：It is asked for described Answers information that topic and option are submitted, the operation information carried out for the answer, for the competing of topic return of guessing The query information guessed information, submitted for guess result.
Such as：If the interactive information includes problem and option, audience interaction information can be that spectators ask for described The answer information that topic and option are submitted.Spectator client can show described problem and choosing in the broadcast interface of target TS fragments , as shown in Figure 6 a.Spectators, can be at the appointed time by clicking the option shown by broadcast interface after seeing this problem Come the answer for selecting oneself to praise.Spectator client can collect the option of spectators' selection, and be obtained according to index file After getting the correct option of this problem, correct option is matched with the selected answer of spectators and judges whether spectators answer with this Correctly, and in broadcast interface output as a result, as shown in Figure 6 b.If at the appointed time, spectator client is not collected into spectators' Selection instruction, that is, spectators do not make a choice at the appointed time, then can be handled by spectators' erroneous answers.
In another example if the interactive information includes answer, audience interaction information can be carried out for the answer Operation information, the operation information can be that spectators click the answer to obtain the click information of answer details.Spectator client The answer can be shown in the broadcast interface of target TS fragments, feel uncertain if spectators have the answer, can click institute The answer in broadcast interface is stated, as shown in Figure 7a.Spectator client is after collecting spectators to the click commands of answer, Ke Yi Broadcast interface shows the details of the answer, as shown in Figure 7b.
In embodiments of the present invention, spectator client is sending video request to server, and receives server return Live video index file after, target TS points can be obtained according to the index for the target TS fragments that the index file includes Piece and target interactive information.It does not need spectator client and interactive information, section is constantly asked to server by the service of timing Resource is saved.
Description based on above method embodiment, the embodiment of the present invention additionally provide a kind of friendship of live data processing method Mutual schematic diagram of a scenario, as shown in Figure 8.
Main broadcaster A, can be by camera assembly typing limb action that main broadcaster's client is configured when carrying out net cast Etc. information, and the microphone typing voice messaging that is configured by main broadcaster's client.The camera assembly and the microphone It can be the component that main broadcaster's client is configured in manufacture, can also be the external equipment for being external in main broadcaster's client. Main broadcaster's client can generate audio-video document after the information for collecting main broadcaster's A typings, and using the audio-video document as The video stream data of live video is sent to server in real time.Server, can be right after receiving the video stream data The video stream data carries out coded treatment, obtains target TS fragments and obtains the address of target TS fragments.
Main broadcaster A, can be by way of enquirement and audience interaction during net cast, such as reads against microphone It goes wrong " professional basketball game summarizes, and scoring a goal can obtain rather ".Main broadcaster's client is in the voice messaging for collecting main broadcaster A Afterwards, the voice messaging can be sent to server as interactive information.Server, can after receiving this interactive information To determine that this interactive information is associated with target TS fragments, i.e., this interactive information is target interactive information.Therefore server is straight It, can be by the ground of this target interactive information and target TS fragments when broadcasting the index for establishing target TS fragments in the index file of video Location is inserted in the index of target TS fragments, and the index file of the index comprising target TS fragments is sent to spectator client.
Spectator client is during live video plays, if detecting that player will be broadcast according to the number of TS fragments Target TS fragments are put, then target TS fragments can be obtained and played according to the index for the target TS fragments that index file includes, and This target interactive information is shown in the broadcast interface of target TS fragments.
The embodiment of the present invention receive main broadcaster's client transmission video stream data and at least one interactive information after, A series of processing can be carried out to video stream data and at least one interactive information, obtain target TS fragments, target TS fragments Address and target interactive information associated with target TS fragments.And target TS is established in the index file of live video The index of the index of fragment, target TS fragments includes address and the target interactive information of target TS fragments；It is broadcast live when receiving When the video request that the spectator client of video is sent, index file is sent to spectator client；Since interactive information is packet It is contained in the index file of live video, which video slicing (TS fragments) would know that according to index file for spectator client Have interactive information, and target interactive information associated with the video slicing can be directly obtained, it can be in live video Interactive information is exported in playing process in time, achievees the purpose that real-time, interactive, improves user experience.
Based on the description of above-mentioned live data processing method embodiment, at a kind of live data Device is managed, which is applied to server, which can be operate in server A computer program (including program code), can also be an entity apparatus comprising in the server.The live streaming number Method shown in Fig. 3 can be executed according to processing unit.Fig. 9 is referred to, the live data processing unit operation such as lower unit：
Receiving unit 101, for receiving the video stream data and at least one friendship that main broadcaster's client of live video is sent Mutual information.
Processing unit 102 obtains the target video point of the live video for being handled the video stream data Piece, and obtain the address of the target video fragment.
Selection unit 103, it is associated with the target video fragment for being chosen from least one interactive information Target interactive information.
Unit 104 is established, the index for establishing the target video fragment in the index file of the live video, The index of the target video fragment includes the address of the target video fragment and the target interactive information.
Output unit 105, for when receive the live video spectator client send video request when, by institute It states index file and is sent to the spectator client, make the spectator client according to target video described in the index file The address for the target video fragment that the index of fragment includes obtains and plays the target video fragment, and in the target The broadcast interface of video slicing shows the target interactive information.
In one embodiment, processing unit 102 can be specifically used for：
Coded treatment is carried out to the video stream data and generates target video fragment；
The target video fragment is stored into content distribution network system；
The address of the target video fragment is obtained from the content distribution network system.
In another embodiment, selection unit 103 can be specifically used for：
Obtain the receiving time for the video stream data that the target video fragment includes；
Obtain the receiving time of at least one interactive information；
According to the receiving time of at least one interactive information, chosen from least one interactive information with it is described The interactive information that the receiving time for the video stream data that target video fragment includes matches；
Selected interactive information is determined as target interactive information associated with the target video fragment.
In another embodiment, establishing unit 104 can be specifically used for：
It is the target video fragment distribution index item in the index file of the live video, the index entry includes Data field and extended field；
The address of the target video fragment is stored into the data field, and the target interactive information is stored To in the extended field.
In another embodiment, when the index of the target video fragment further includes the broadcasting of the target video fragment Long, the playing duration is less than the first pre-set length threshold.
According to one embodiment of present invention, each step involved by method shown in Fig. 3 may each be as shown in Figure 9 Live data processing unit in each unit be performed.For example, step S101, S102, S103 shown in Fig. 3, S104, S105 can respectively receiving unit 101, processing unit 102 shown in Fig. 9, selection unit 103, establish unit 104 It is executed with output unit 105.
According to another embodiment of the invention, each unit in live data processing unit shown in Fig. 9 can divide Or all one or several other units are not merged into constitute or some (a little) unit therein can also be split again It is constituted for functionally smaller multiple units, this may be implemented similarly to operate, the skill without influencing the embodiment of the present invention The realization of art effect.Said units are that logic-based function divides, and in practical applications, the function of a unit can also be by Multiple units are realized or the function of multiple units is realized by a unit.In other embodiments of the invention, number is broadcast live Can also include other units according to processing unit, in practical applications, these functions can also be assisted to realize by other units, and And it can be cooperated and be realized by multiple units.
It according to another embodiment of the invention, can be by including central processing unit (CPU), random access memory It is transported on the universal computing device of such as computer of the processing elements such as medium (RAM), read-only storage medium (ROM) and memory element Row is able to carry out the computer program (including program code) of each step involved by correlation method as shown in Figure 3, carrys out structure Live data processing unit equipment as shown in Figure 9 is made, and to realize the live data processing method of the embodiment of the present invention. The computer program can be recorded on such as computer readable recording medium storing program for performing, and be loaded by computer readable recording medium storing program for performing In above-mentioned computing device, and run wherein.
The video stream data and at least one friendship that the embodiment of the present invention is sent in the main broadcaster's client for receiving live video After mutual information, a series of processing can be carried out to video stream data and at least one interactive information, obtain target video point Piece, the address of target video fragment and target interactive information associated with target video fragment.And in the rope of live video The index of target video fragment is established in quotation part, the index of the target video fragment includes address and the friendship of target video fragment Mutual information；When the video request that the spectator client for receiving the live video is sent, the index file is sent to The spectator client；Since interactive information is included in the index file of live video, spectator client is according to index File would know which video slicing (TS fragments) has interactive information, and can directly obtain associated with the video slicing Target interactive information, achieve the purpose that real-time, interactive, improve user experience.
Based on the description of above-mentioned live data processing method embodiment, another embodiment of the present invention also discloses a kind of live streaming Data processing equipment, the live data processing unit are applied to the spectator client of live video, the live data processing unit The computer program (including program code) that can be operate in terminal can also be comprising a reality in the terminal Body device.The live data processing unit can execute method shown in Fig. 4.Figure 10 is referred to, the live data processing unit Operation such as lower unit：
Request unit 201, for sending video request to server.
Receiving unit 202, the index file for receiving the live video that the server returns, the index file packet Include the index that the server is established by the target video fragment of the live video, the index packet of the target video fragment Include the address of the target video fragment and target interactive information associated with the target video fragment；Wherein, the mesh The video stream data that mark video slicing sends main broadcaster's client of the live video by the server is handled to obtain； The address of the target video fragment is acquired by the server；The target interactive information is by the server from described It chooses and obtains at least one interactive information that main broadcaster's client is sent.
Reading unit 203, described in the index of the target video fragment for including according to the index file is read The address of target video fragment and target interactive information.
Processing unit 204, for the target video fragment to be obtained and played according to the address of the target video fragment, And show the target interactive information in the broadcast interface of the target video fragment.
In one embodiment, the index file includes the index entry of the target video fragment, the index Item includes data field and extended field；
The address of the target video fragment is stored in the data field, and the target interactive information is stored in In the extended field.
In another embodiment, reading unit 203 can be specifically used for：
The index entry of the target video fragment is searched in the index file；
Obtain the address of the target video fragment from the data field of the index entry, and from the index entry The target interactive information is obtained in extended field.
In another embodiment, processing unit 204 can be specifically used for：
The target video fragment is obtained from content distribution network system according to the address of the target video fragment；
Play the target video fragment.
In another embodiment, processing unit 204 can be specifically used for：
The target video fragment is loaded onto buffering area, the length of the buffering area is less than the second pre-set length threshold；
Play the target video fragment that the buffering area is loaded.
In another embodiment, the interactive information includes following at least one：Problem and option, answer, guess topic Mesh, guess result；Processing unit 204, it may also be used for：
Collect audience interaction information of the live video in playing process, and to the audience interaction information into industry Business is handled；
The audience interaction information includes following at least one：The answer information submitted for described problem and option, It is submitted for the operation information of answer progress, for the quiz information of the guess topic return, for guess result Query information.
According to one embodiment of present invention, each step involved by method shown in Fig. 4 may each be by Figure 10 institutes Each unit in the live data processing unit shown is performed.For example, step S201, S202, S203 shown in Fig. 4, S204 can respectively request unit 201, receiving unit 202, reading unit 203 and processing unit 204 shown in Figure 10 It executes.
According to another embodiment of the invention, each unit in live data processing unit shown in Fig. 10 can divide Or all one or several other units are not merged into constitute or some (a little) unit therein can also be split again It is constituted for functionally smaller multiple units, this may be implemented similarly to operate, the skill without influencing the embodiment of the present invention The realization of art effect.Said units are that logic-based function divides, and in practical applications, the function of a unit can also be by Multiple units are realized or the function of multiple units is realized by a unit.In other embodiments of the invention, number is broadcast live Can also include other units according to processing unit, in practical applications, these functions can also be assisted to realize by other units, and And it can be cooperated and be realized by multiple units.
It according to another embodiment of the invention, can be by including central processing unit (CPU), random access memory It is transported on the universal computing device of such as computer of the processing elements such as medium (RAM), read-only storage medium (ROM) and memory element Row is able to carry out the computer program (including program code) of each step involved by correlation method as shown in Figure 4, carrys out structure Live data processing unit equipment as shown in Figure 10 is made, and to realize the live data processing side of the embodiment of the present invention Method.The computer program can be recorded on such as computer readable recording medium storing program for performing, and pass through computer readable recording medium storing program for performing It is loaded into above-mentioned computing device, and runs wherein.
In embodiments of the present invention, spectator client is sending video request to server, and receives server return Live video index file after, target TS points can be obtained according to the index for the target TS fragments that the index file includes Piece and target interactive information associated with target TS fragments.Service of the spectator client by timing is not needed constantly to clothes Business device asks interactive information, saves resource.
Based on method and device shown in above-described embodiment, the embodiment of the present invention additionally provides a kind of server, the clothes Business device can be used for being broadcast live data processing.Figure 11 is referred to, the internal structure of the server includes at least encoder 301, place Manage device 302, communication interface 303 and computer storage media 304.Wherein, the encoder 301 in server, processor 302, logical Believe that interface 303 and computer storage media 304 can be connected by bus or other modes, in Figure 11 shown in the embodiment of the present invention For being connected by bus 305.
Communication interface 303 is to realize to interact between server and main broadcaster's client or spectator client to hand over information The medium changed.Processor 302 (or CPU (Central Processing Unit, central processing unit)) is the calculating of server Core and control core are adapted for carrying out one or one or more instruction, be particularly adapted to load and execute one or one with Upper instruction is to realize correlation method flow or corresponding function.Encoder 301 can be used for being regarded according to collected described in server Frequency flow data carries out coded treatment and generates TS fragments.
The embodiment of the present invention can acquire the main broadcaster visitor of live video in specific implementation process by communication interface 303 The video stream data and at least one interactive information that family end is sent, compile the video stream data by encoder 301 Code processing generates target TS fragments, and the address of target TS fragments is obtained by processor 302, is selected from least one interactive information Target interactive information associated with target TS fragments is taken, the rope of target TS fragments is established in the index file of live video Draw, the index of target video fragment includes address and the target interactive information of target video fragment；It is regarded when receiving the live streaming When the video request that the spectator client of frequency is sent, the index file is sent to described spectator client, etc..
Computer storage media (Memory) 304 is the memory device in server, for storing program and data.It can be with Understand, computer storage media herein both may include the built-in storage medium of server, naturally it is also possible to including clothes The expansion storage medium that business device is supported.Computer storage media provides memory space, which stores server Operating system.Also, it is also housed suitable for by processor loads and executes one or more than one in the memory space Instruction, these instructions can be one or more computer program (including program code).It should be noted that herein Computer storage media can be high-speed RAM memory, can also be non-labile memory (non-volatile Memory), a for example, at least magnetic disk storage；It optionally can also be at least one calculating for being located remotely from aforementioned processor Machine storage medium.
In embodiments of the present invention, processor 302 loads and executes one stored in computer storage media or one Above instructions, to realize the corresponding steps of method flow shown in above-mentioned Fig. 3；In the specific implementation, one in computer storage media Item or one or more first instruction are loaded by processor 302 and execute following steps：
Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
The video stream data is handled to obtain the target video fragment of the live video, and obtains the target The address of video slicing；
Target interactive information associated with the target video fragment is chosen from least one interactive information；
The index of the target video fragment, the target video fragment are established in the index file of the live video Index include the address of the target video fragment and the target interactive information；
When the video request that the spectator client for receiving the live video is sent, the index file is sent to The spectator client makes the index for the target video fragment that the spectator client includes according to the index file obtain The target video fragment is taken and played, and the target interactive information is shown in the broadcast interface of the target video fragment.
In one embodiment, the video stream data is being handled to obtain the target video of the live video Fragment, and when obtaining the address of the target video fragment, this or one or more first instruction are loaded by the processor 302 And it executes：
Coded treatment is carried out to the video stream data and generates target video fragment；
The target video fragment is stored into content distribution network system；
The address of the target video fragment is obtained from the content distribution network system.
It is related to the target video fragment being chosen from least one interactive information in another embodiment The target interactive information of connection, this or one or more first instruction by the processor 302 load and execute：
Obtain the receiving time for the video stream data that the target video fragment includes；
Obtain the receiving time of at least one interactive information；
According to the receiving time of at least one interactive information, chosen from least one interactive information with it is described The interactive information that the receiving time for the video stream data that target video fragment includes matches；
Selected interactive information is determined as target interactive information associated with the target video fragment.
In another embodiment, the index of the target video fragment is established in the index file of the live video When, this or one or more first instruction by the processor 302 load and execute：
It is the target video fragment distribution index item in the index file of the live video, the index entry includes Data field and extended field；
The address of the target video fragment is stored into the data field, and the target interactive information is stored To in the extended field.
In another embodiment, when the index of the target video fragment further includes the broadcasting of the target video fragment Long, the playing duration is less than the first pre-set length threshold.
The video stream data and at least one friendship that the embodiment of the present invention is sent in the main broadcaster's client for receiving live video After mutual information, a series of processing can be carried out to video stream data and at least one interactive information, obtain target video point Piece, the address of target video fragment and target interactive information associated with target video fragment.And in the rope of live video The index of target video fragment is established in quotation part, the index of the target video fragment includes address and the mesh of target video fragment Mark interactive information；When the video request that the spectator client for receiving the live video is sent, the index file is sent out Give the spectator client；Since interactive information is included in the index file of live video, spectator client foundation Index file would know which video slicing (TS fragments) has interactive information, and can directly obtain and the video slicing phase Associated target interactive information achievees the purpose that real-time, interactive, improves user experience.
Description based on above method embodiment and device embodiment, the embodiment of the present invention also provide a kind of terminal, should Terminal can be used for being broadcast live data processing.The terminal operating has the spectator client of live video.Figure 12 is referred to, it is described Terminal inner structure includes at least decoder 401, processor 402, input equipment 403, output equipment 404 and computer storage Medium 405.Wherein, the decoder 401 in terminal, processor 402, input equipment 403, output equipment 404 and computer are deposited Storage media 405 can be connected by bus or other modes, to be connected as by bus 406 in Figure 12 shown in the embodiment of the present invention Example.
The embodiment of the present invention can send video request by output equipment 404 in specific implementation process to server, The index file for the live video that server returns is received by input equipment 403, and by processor 402 according to the index The index for the target video fragment that file includes reads address and the target interactive information of target video fragment；And according to target Video slicing volume address obtains target video fragment.Target video fragment is decoded by decoder 401 again, after the completion of decoding Show that the target is handed over by the target video fragment after 402 broadcast decoder of processor, and in the broadcast interface of target video fragment Mutual information.
Computer storage media 405 can be used for storing computer program, and the computer program includes program instruction, described Processor is used to execute the program instruction of the computer storage media storage.The computer storage media is the note in terminal Equipment is recalled, for storing program and data.It is understood that computer storage media herein both may include in terminal Built-in storage medium, naturally it is also possible to the expansion storage medium supported including terminal.It is empty that computer storage media provides storage Between, which stores the operating system of terminal.Also, it is also housed in the memory space suitable for being loaded by processor And the one or more than one instructions executed, these instructions can be one or more computer program (including journeys Sequence code).It should be noted that computer storage media herein can be high-speed RAM memory, can also be non-unstable Memory (non-volatile memory), a for example, at least magnetic disk storage；It optionally can also be at least one position In the computer storage media far from aforementioned processor.
In one embodiment, it can be loaded by processor 402 and execute one or one stored in computer storage media Above instructions, to realize the corresponding steps of method shown in Fig. 4；In the specific implementation, one in computer storage media or One or more second instruction is loaded by processor 402 and executes following steps：
The spectator client of live video sends video request to server；
The index file for the live video that the server returns is received, the index file includes the server The index of the index established by the target video fragment of the live video, the target video fragment includes that the target regards The address of frequency division piece and target interactive information associated with the target video fragment；Wherein, the target video fragment by The video stream data that the server sends main broadcaster's client of the live video is handled to obtain；The target video The address of fragment is acquired by the server；The target interactive information is sent out by the server from main broadcaster's client It chooses and obtains at least one interactive information sent；
The index for the target video fragment for including according to the index file reads the ground of the target video fragment Location and the target interactive information；
It is obtained according to the address of the target video fragment and plays the target video fragment, and in the target video The broadcast interface of fragment shows the target interactive information.
In one embodiment, the index file includes the index entry of the target video fragment, the index Item includes data field and extended field；
The address of the target video fragment is stored in the data field, and the target interactive information is stored in In the extended field.
In another embodiment, institute is read in the index for the target video fragment for including according to the index file When stating the address of target video fragment and the target interactive information, this or one or more second instruction by the processor 402 load and execute：
The index entry of the target video fragment is searched in the index file；
Obtain the address of the target video fragment from the data field of the index entry, and from the index entry The target interactive information is obtained in extended field.
In another embodiment, is being obtained according to the address of the target video fragment and playing the target video point When piece, this or one or more second instruction by the processor 402 load and execute：
The target video fragment is obtained from content distribution network system according to the address of target video fragment；
Play the target video fragment.
In another embodiment, when playing the target video fragment, this or one or more second instruction by The processor 402 is loaded and is executed：
Target video fragment is loaded onto buffering area, the length of the buffering area is less than the second pre-set length threshold；
Play the target video fragment that the buffering area is loaded.
In another embodiment, interactive information includes following at least one：It is problem and option, answer, guess topic, competing Guess result；This or one or more second instruction can also by the processor 402 load and execute：
Collect audience interaction information of the live video in playing process, and to the audience interaction information into industry Business is handled；
The audience interaction information includes following at least one：The answer information submitted for described problem and option, It is submitted for the operation information of answer progress, for the quiz information of the guess topic return, for guess result Query information.
In embodiments of the present invention, spectator client is sending video request to server, and receives server return Live video index file after, target TS points can be obtained according to the index for the target TS fragments that the index file includes Piece and target interactive information associated with target TS fragments.Service of the spectator client by timing is not needed constantly to clothes Business device asks interactive information, saves resource.
The above disclosure is only the preferred embodiments of the present invention, cannot limit the right model of the present invention with this certainly It encloses, therefore equivalent changes made in accordance with the claims of the present invention, is still within the scope of the present invention.

Claims (17)

1. a kind of live data processing method, which is characterized in that including：
Receive the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
The video stream data is handled to obtain the target video fragment of the live video, and obtains the target video The address of fragment；
Target interactive information associated with the target video fragment is chosen from least one interactive information；
The index of the target video fragment, the rope of the target video fragment are established in the index file of the live video Draw the address including the target video fragment and the target interactive information；
When the video request that the spectator client for receiving the live video is sent, the index file is sent to described Spectator client makes described in the spectator client according to the index of target video fragment described in the index file includes The address of target video fragment obtains and plays the target video fragment, and aobvious in the broadcast interface of the target video fragment Show the target interactive information.
2. the method as described in claim 1, which is characterized in that it is described the video stream data is handled to obtain it is described straight The target video fragment of video is broadcast, and obtains the address of the target video fragment, including：
Coded treatment is carried out to the video stream data and generates target video fragment；
The target video fragment is stored into content distribution network system；
The address of the target video fragment is obtained from the content distribution network system.
3. the method as described in claim 1, which is characterized in that it is described from least one interactive information choose with it is described The associated target interactive information of target video fragment, including：
Obtain the receiving time for the video stream data that the target video fragment includes；
Obtain the receiving time of at least one interactive information；
According to the receiving time of at least one interactive information, chosen and the target from least one interactive information The interactive information that the receiving time for the video stream data that video slicing includes matches；
Selected interactive information is determined as target interactive information associated with the target video fragment.
4. the method as described in claim 1, which is characterized in that it is described in the index file of the live video establish described in The index of target video fragment, including：
It is the target video fragment distribution index item in the index file of the live video, the index entry includes data Field and extended field；
The address of the target video fragment is stored into the data field, and the target interactive information is stored to institute It states in extended field.
5. method according to any one of claims 1-4, which is characterized in that the index of the target video fragment further includes institute State the playing duration of target video fragment；The playing duration is less than the first pre-set length threshold.
6. a kind of live data processing method, which is characterized in that including：
The spectator client of live video sends video request to server；
Receive the index file for the live video that the server returns, the index file includes the server for institute The index that the target video fragment of live video is established is stated, the index of the target video fragment includes the target video point The address of piece and target interactive information associated with the target video fragment；Wherein, the target video fragment is by described The video stream data that server sends main broadcaster's client of the live video is handled to obtain；The target video fragment Address acquired by the server；The target interactive information is sent by the server from main broadcaster's client It chooses and obtains at least one interactive information；
Read in the index for the target video fragment for including from the index file target video fragment address and The target interactive information；
The target video fragment is obtained and played according to the address of the target video fragment, and in the target video fragment Broadcast interface show the target interactive information.
7. method as claimed in claim 6, which is characterized in that the index file includes the rope of the target video fragment Draw item, the index entry includes data field and extended field；
The address of the target video fragment is stored in the data field, and the target interactive information is stored in described In extended field.
8. the method for claim 7, which is characterized in that the target video for including according to the index file The index of fragment reads the address of the target video fragment and the target interactive information, including：
The index entry of the target video fragment is searched in the index file；
The address of the target video fragment, and the extension from the index entry are obtained from the data field of the index entry The target interactive information is obtained in field.
9. such as claim 6-8 any one of them methods, which is characterized in that the address according to the target video fragment It obtains and plays the target video fragment and include：
The target video fragment is obtained from content distribution network system according to the address of the target video fragment；
Play the target video fragment.
10. method as claimed in claim 9, which is characterized in that the broadcasting target video fragment, including：
The target video fragment is loaded onto buffering area, the length of the buffering area is less than the second pre-set length threshold；
Play the target video fragment that the buffering area is loaded.
11. method as claimed in claim 6, which is characterized in that the method further includes：
Audience interaction information of the live video in playing process is collected, and the audience interaction information is carried out at business Reason；
Wherein, the interactive information includes following at least one：Problem and option, answer, guess topic, guess result；It is described Audience interaction information includes following at least one：The answer information submitted for described problem and option is directed to the answer The operation information of progress, for it is described guess topic return quiz information, for guess result submit query information.
12. a kind of live data processing unit is applied to server, which is characterized in that including：
Receiving unit, for receiving the video stream data and at least one interactive information that main broadcaster's client of live video is sent；
Processing unit obtains the target video fragment of the live video for being handled the video stream data, and obtains Take the address of the target video fragment；
Selection unit is handed over for choosing target associated with the target video fragment from least one interactive information Mutual information；
Unit is established, the index for establishing the target video fragment in the index file of the live video, the mesh The index for marking video slicing includes the address of the target video fragment and the target interactive information；
Output unit, for when receive the live video spectator client send video request when, by the index File is sent to the spectator client, makes the spectator client according to target video fragment described in the index file The address for the target video fragment that index includes obtains and plays the target video fragment, and in the target video point The broadcast interface of piece shows the target interactive information.
13. a kind of live data processing unit is applied to the spectator client of live video, which is characterized in that including：
Request unit, for sending video request to server；
Receiving unit, the index file for receiving the live video that the server returns, the index file include The index of the index that the server is established by the target video fragment of the live video, the target video fragment includes The address of the target video fragment and target interactive information associated with the target video fragment；Wherein, the target The video stream data that video slicing sends main broadcaster's client of the live video by the server is handled to obtain；Institute The address for stating target video fragment is acquired by the server；The target interactive information is by the server from the master It broadcasts to choose at least one interactive information of client transmission and obtain；
Reading unit reads the target video in the index of the target video fragment for including from the index file The address of fragment and the target interactive information；
Processing unit, for being obtained according to the address of the target video fragment and playing the target video fragment, and in institute The broadcast interface for stating target video fragment shows the target interactive information.
14. a kind of server for being broadcast live data processing, which is characterized in that including：
Processor is adapted for carrying out one or one or more instruction；And
Computer storage media, the computer storage media is stored with one or one or more is instructed, described one or one Above instructions are suitable for being loaded by the processor and executing live data processing method as described in any one in claim 1-5.
15. a kind of terminal for being broadcast live data processing, the terminal operating has the spectator client of live video, described Terminal includes input equipment and output equipment, which is characterized in that further includes：
Processor is adapted for carrying out one or one or more instruction；And
Computer storage media, the computer storage media is stored with one or one or more is instructed, described one or one Above instructions are suitable for being loaded by the processor and being executed such as claim 6-11 any one of them live data processing methods.
16. a kind of computer storage media, which is characterized in that the computer storage media be stored with one article or one or more One instruction, described one or one or more first instruction be suitable for loaded by processor and executed such as any one of claim 1-5 institutes The live data processing method stated.
17. a kind of computer storage media, which is characterized in that the computer storage media be stored with one article or one or more Two instruction, described one or one or more second instruction be suitable for loaded by processor and executed such as any one of claim 6-11 institutes The live data processing method stated.
CN201810388944.9A 2018-04-26 2018-04-26 Live broadcast data processing method and device, server, terminal and storage medium Active CN108391179B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810388944.9A CN108391179B (en) 2018-04-26 2018-04-26 Live broadcast data processing method and device, server, terminal and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810388944.9A CN108391179B (en) 2018-04-26 2018-04-26 Live broadcast data processing method and device, server, terminal and storage medium

Publications (2)

Publication Number Publication Date
CN108391179A true CN108391179A (en) 2018-08-10
CN108391179B CN108391179B (en) 2021-07-27

Family

ID=63064967

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810388944.9A Active CN108391179B (en) 2018-04-26 2018-04-26 Live broadcast data processing method and device, server, terminal and storage medium

Country Status (1)

Country Link
CN (1) CN108391179B (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109600678A (en) * 2018-12-19 2019-04-09 北京达佳互联信息技术有限公司 Information displaying method, apparatus and system, server, terminal, storage medium
CN109960731A (en) * 2019-03-28 2019-07-02 腾讯音乐娱乐科技（深圳）有限公司 A kind of data processing method, equipment and storage medium
CN110290392A (en) * 2019-06-28 2019-09-27 广州酷狗计算机科技有限公司 Live information display methods, device, equipment and storage medium
CN110636340A (en) * 2019-09-30 2019-12-31 普联技术有限公司 Video file uploading method and storage device
CN110636335A (en) * 2019-09-24 2019-12-31 江苏鸿信***集成有限公司 HLS (HTTP live streaming) quick sharing method for real-time video
CN111182328A (en) * 2020-02-12 2020-05-19 北京达佳互联信息技术有限公司 Video editing method, device, server, terminal and storage medium
CN111526430A (en) * 2020-03-26 2020-08-11 王东 File matching method and device and electronic equipment
CN111787382A (en) * 2020-07-03 2020-10-16 北京英迈琪科技有限公司 Data interaction method and system
CN112261118A (en) * 2020-10-19 2021-01-22 腾讯科技（深圳）有限公司 Multimedia data anomaly detection method, terminal and server
CN112423012A (en) * 2020-11-18 2021-02-26 青岛华升联信智慧科技有限公司 Multistage-load live broadcast method
CN112969093A (en) * 2019-12-13 2021-06-15 腾讯科技（北京）有限公司 Interactive service processing method, device, equipment and storage medium
CN113438520A (en) * 2021-06-29 2021-09-24 北京奇艺世纪科技有限公司 Data processing method, device and system
CN113742519A (en) * 2021-08-31 2021-12-03 杭州登虹科技有限公司 Multi-object storage cloud video Timeline storage method and system
CN114025201A (en) * 2021-10-29 2022-02-08 恒安嘉新（北京）科技股份公司 Video playing method, device, equipment and storage medium
CN114071174A (en) * 2021-11-15 2022-02-18 咪咕文化科技有限公司 Processing method of live content, electronic equipment and readable storage medium
CN114189699A (en) * 2020-09-15 2022-03-15 阿里巴巴集团控股有限公司 Government affair service information providing method and device and electronic equipment
CN114302218A (en) * 2021-12-29 2022-04-08 北京力拓飞远科技有限公司 Interactive video generation method, system and storage medium
CN114697746A (en) * 2020-12-28 2022-07-01 北京金山云网络技术有限公司 Video playing starting method and device, electronic equipment and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632332A (en) * 2013-11-29 2014-03-12 腾讯科技(成都)有限公司 Subject question and answer method, device and system
CN104683884A (en) * 2015-02-09 2015-06-03 网宿科技股份有限公司 Live method and system for stream medium
EP2988513A1 (en) * 2014-08-20 2016-02-24 Xiaomi Inc. Method for providing real-time video and device thereof as well as server and terminal device
CN106028137A (en) * 2016-06-22 2016-10-12 北京小米移动软件有限公司 Live streaming processing method and apparatus
CN106162230A (en) * 2016-07-28 2016-11-23 北京小米移动软件有限公司 The processing method of live information, device, Zhu Boduan, server and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632332A (en) * 2013-11-29 2014-03-12 腾讯科技(成都)有限公司 Subject question and answer method, device and system
EP2988513A1 (en) * 2014-08-20 2016-02-24 Xiaomi Inc. Method for providing real-time video and device thereof as well as server and terminal device
CN104683884A (en) * 2015-02-09 2015-06-03 网宿科技股份有限公司 Live method and system for stream medium
CN106028137A (en) * 2016-06-22 2016-10-12 北京小米移动软件有限公司 Live streaming processing method and apparatus
CN106162230A (en) * 2016-07-28 2016-11-23 北京小米移动软件有限公司 The processing method of live information, device, Zhu Boduan, server and system

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109600678A (en) * 2018-12-19 2019-04-09 北京达佳互联信息技术有限公司 Information displaying method, apparatus and system, server, terminal, storage medium
CN109600678B (en) * 2018-12-19 2021-11-02 北京达佳互联信息技术有限公司 Information display method, device and system, server, terminal and storage medium
CN109960731A (en) * 2019-03-28 2019-07-02 腾讯音乐娱乐科技（深圳）有限公司 A kind of data processing method, equipment and storage medium
CN109960731B (en) * 2019-03-28 2022-11-18 腾讯音乐娱乐科技（深圳）有限公司 Data processing method, device and storage medium
CN110290392A (en) * 2019-06-28 2019-09-27 广州酷狗计算机科技有限公司 Live information display methods, device, equipment and storage medium
CN110636335A (en) * 2019-09-24 2019-12-31 江苏鸿信***集成有限公司 HLS (HTTP live streaming) quick sharing method for real-time video
CN110636340A (en) * 2019-09-30 2019-12-31 普联技术有限公司 Video file uploading method and storage device
CN110636340B (en) * 2019-09-30 2021-11-09 普联技术有限公司 Video file uploading method, storage device, terminal device and storage medium
CN112969093A (en) * 2019-12-13 2021-06-15 腾讯科技（北京）有限公司 Interactive service processing method, device, equipment and storage medium
CN112969093B (en) * 2019-12-13 2023-09-08 腾讯科技（北京）有限公司 Interactive service processing method, device, equipment and storage medium
CN111182328B (en) * 2020-02-12 2022-03-25 北京达佳互联信息技术有限公司 Video editing method, device, server, terminal and storage medium
CN111182328A (en) * 2020-02-12 2020-05-19 北京达佳互联信息技术有限公司 Video editing method, device, server, terminal and storage medium
CN111526430A (en) * 2020-03-26 2020-08-11 王东 File matching method and device and electronic equipment
CN111787382A (en) * 2020-07-03 2020-10-16 北京英迈琪科技有限公司 Data interaction method and system
CN114189699A (en) * 2020-09-15 2022-03-15 阿里巴巴集团控股有限公司 Government affair service information providing method and device and electronic equipment
CN112261118A (en) * 2020-10-19 2021-01-22 腾讯科技（深圳）有限公司 Multimedia data anomaly detection method, terminal and server
CN112423012A (en) * 2020-11-18 2021-02-26 青岛华升联信智慧科技有限公司 Multistage-load live broadcast method
CN112423012B (en) * 2020-11-18 2023-05-09 青岛华升联信智慧科技有限公司 Multi-stage load live broadcast method
CN114697746A (en) * 2020-12-28 2022-07-01 北京金山云网络技术有限公司 Video playing starting method and device, electronic equipment and system
CN113438520A (en) * 2021-06-29 2021-09-24 北京奇艺世纪科技有限公司 Data processing method, device and system
CN113742519A (en) * 2021-08-31 2021-12-03 杭州登虹科技有限公司 Multi-object storage cloud video Timeline storage method and system
CN114025201A (en) * 2021-10-29 2022-02-08 恒安嘉新（北京）科技股份公司 Video playing method, device, equipment and storage medium
CN114071174A (en) * 2021-11-15 2022-02-18 咪咕文化科技有限公司 Processing method of live content, electronic equipment and readable storage medium
CN114302218A (en) * 2021-12-29 2022-04-08 北京力拓飞远科技有限公司 Interactive video generation method, system and storage medium

Also Published As

Publication number Publication date
CN108391179B (en) 2021-07-27

Similar Documents

Publication Publication Date Title
CN108391179A (en) live data processing method, device, server, terminal and storage medium
CN108184144B (en) Live broadcast method and device, storage medium and electronic equipment
CN103548358B (en) Receiving device, method of reseptance and information providing apparatus
KR101817652B1 (en) Method and apparatus for recieving game streamming data and method and server of transmitting game streaming data
CN104126175B (en) Virtualization in self adaptation stream establishment and conveying
CN100385424C (en) Information processing apparatus and content information processing method
CN105635764B (en) Method and device for playing push information in live video
CN109891896A (en) Anchor for live stream
CN110267053A (en) Live broadcasting method, apparatus and system
CN109565620A (en) Low latency HTTP real-time streaming transport
CN110191372A (en) Multimedia interaction method, system and device
CN108462883B (en) A kind of living broadcast interactive method, apparatus, terminal device and storage medium
CN108810052B (en) Live broadcast server selection method and device
US20140143437A1 (en) Converting Live Streaming Content to Video-on-Demand Streaming Content
US10864447B1 (en) Highlight presentation interface in a game spectating system
CN109327741A (en) Game live broadcasting method, device and system
CN109829064B (en) Media resource sharing and playing method and device, storage medium and electronic device
CN104980500B (en) A kind of information display method and terminal
CN113630630B (en) Method, device and equipment for processing video comment dubbing information
CN105472401B (en) The method and system of advertisement are played during network direct broadcasting
US10363488B1 (en) Determining highlights in a game spectating system
WO2015043415A1 (en) Method, device and system for video content interaction
CN112019905A (en) Live broadcast playback method, computer equipment and readable storage medium
CN104348899A (en) Client and server multimedia content sharing method and sharing system
CN105516741A (en) Method and apparatus for inserting game in live broadcast interface

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant