CN110858923B - Method and device for generating segmented media file and storage medium - Google Patents
Method and device for generating segmented media file and storage medium Download PDFInfo
- Publication number
- CN110858923B CN110858923B CN201810974160.4A CN201810974160A CN110858923B CN 110858923 B CN110858923 B CN 110858923B CN 201810974160 A CN201810974160 A CN 201810974160A CN 110858923 B CN110858923 B CN 110858923B
- Authority
- CN
- China
- Prior art keywords
- binary data
- file
- data
- buffer
- different types
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 239000000872 buffer Substances 0.000 claims abstract description 63
- 238000004458 analytical method Methods 0.000 claims abstract description 35
- 238000007405 data analysis Methods 0.000 claims description 15
- 230000008878 coupling Effects 0.000 claims description 5
- 238000010168 coupling process Methods 0.000 claims description 5
- 238000005859 coupling reaction Methods 0.000 claims description 5
- 230000003139 buffering effect Effects 0.000 claims 3
- 230000008569 process Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 22
- 238000000605 extraction Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 238000004806 packaging method and process Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 2
- AWSBQWZZLBPUQH-UHFFFAOYSA-N mdat Chemical compound C1=C2CC(N)CCC2=CC2=C1OCO2 AWSBQWZZLBPUQH-UHFFFAOYSA-N 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure provides a method for generating a segmented media file, which comprises the steps of obtaining binary data included in a streaming media format file, and caching the obtained binary data in a buffer; extracting corresponding binary data from the buffer by an analyzer for analysis, and caching the analysis result of the binary data into the buffer; and coding the analysis result of the binary data in the buffer by an encoder to form a corresponding segmented media file. The disclosure also provides a segmented media file generating device and a storage medium.
Description
Technical Field
The present disclosure relates to multimedia technologies, and in particular, to a method and an apparatus for generating a segmented media file, and a storage medium.
Background
In the related art, the browser does not support playing of a streaming media Format (FLV) media file, and can play by using a Video element of the browser by parsing binary data in the FLV file and encoding the binary data into a segmented MP4(FMP4, segmented MP4) file.
However, in the prior art, binary data in a streaming media format file is parsed in a serial manner by a plurality of parsers, that is, parsed data and un-parsed data are transmitted to a next parser for further parsing, and parsing efficiency is low.
Disclosure of Invention
In view of this, embodiments of the present disclosure provide a method, an apparatus, and a storage medium for generating a segmented media file, which can efficiently parse corresponding binary data from a file in a streaming media format, and encode the binary data according to the parsed binary data to form a corresponding segmented media file
The technical scheme of the embodiment of the disclosure is realized as follows:
the embodiment of the disclosure provides a method for generating a segmented media file, which includes:
acquiring binary data included in the streaming media format file, and caching the acquired binary data in a buffer;
extracting corresponding binary data from the buffer by an analyzer for analysis, and caching the analysis result of the binary data into the buffer;
and coding the analysis result of the binary data in the buffer by an encoder to form a corresponding segmented media file.
In the above solution, the extracting, by the parser, the corresponding binary data from the buffer for parsing includes:
extracting binary data from the buffer correspondingly in a parallel mode through resolvers of different types for resolution;
the different types of resolvers include at least one of:
a metadata information analyzer, an audio analyzer and a video analyzer.
In the above solution, the extracting, by the parser, the corresponding binary data from the buffer for parsing includes:
and respectively coupling the unified cache structures through resolvers of different types, and extracting binary data corresponding to the resolver types from the cache through resolvers of different types by using the unified cache structures for resolution.
In the above scheme, the caching the parsing result of the binary data into the buffer includes:
caching the analysis results of the binary data corresponding to the analyzers into the cache by using the unified cache structure through different types of analyzers;
and sharing the analysis results of the respective binary data by the different types of analyzers and by using the uniform cache structure.
In the above scheme, the method further comprises:
and when the resolver needs to resolve the corresponding binary data based on the resolving results of resolvers of other types, obtaining the resolving result of the corresponding binary data through the unified cache structure.
In the above scheme, the method further comprises:
setting a corresponding type of resolver according to the binary data type included in the acquired streaming media format file;
initializing the set resolver of the corresponding type and a buffer corresponding to the resolver.
In the foregoing solution, the obtaining binary data included in the streaming media format file includes:
and analyzing the nesting structure of the streaming media format file, and reading corresponding binary data according to the nesting structure.
In the above scheme, the method further comprises:
and sending the segmented media file to a media element of the webpage for playing through a media source expansion interface of the webpage.
An embodiment of the present disclosure further provides a device for generating a segmented media file, including:
the data acquisition module is used for acquiring binary data included in the streaming media format file;
the data acquisition module is used for caching the acquired binary data in a buffer;
the data analysis module is used for extracting corresponding binary data from the buffer memory through an analyzer for analysis, and caching the analysis result of the binary data into the buffer memory;
and the data analysis module is used for coding the analysis result of the binary data in the buffer through an encoder so as to form a corresponding segmented media file.
In the above-mentioned scheme, the first step of the method,
the data analysis module is used for extracting binary data from the buffer correspondingly in a parallel mode through resolvers of different types for analysis;
the different types of resolvers include at least one of:
a metadata information analyzer, an audio analyzer and a video analyzer.
In the above-mentioned scheme, the first step of the method,
the data analysis module is used for respectively coupling the uniform cache structures through different types of analyzers;
and the data analysis module is used for extracting binary data corresponding to the types of the resolvers from the buffer by using the unified buffer structure through the resolvers of different types for analysis.
In the above-mentioned scheme, the first and second light sources,
the data analysis module is used for caching the analysis results of the binary data corresponding to the analyzers into the cache by using the unified cache structure through the analyzers of different types;
and the data analysis module is used for sharing the respective analysis results of the corresponding binary data by using the uniform cache structure through the resolvers of different types.
In the above scheme, the data parsing module is configured to obtain a parsing result of corresponding binary data through the unified cache structure.
In the above scheme, the apparatus further comprises:
the initialization module is used for setting a corresponding type of resolver according to the binary data type included in the acquired streaming media format file;
the initialization module is used for initializing the set resolver of the corresponding type and the buffer corresponding to the resolver.
In the above scheme, the data obtaining module is configured to analyze a nested structure of the streaming media format file, and read corresponding binary data according to the nested structure.
In the above solution, the apparatus further includes:
and the data receiving and sending module is used for sending the segmented media file to the media element of the webpage for playing through the media source expansion interface of the webpage.
The embodiment of the present disclosure further provides a device for generating a segmented media file, including:
a memory for storing executable instructions;
and the processor is used for realizing the generation method of the segmented media file provided by the embodiment of the disclosure when the executable instruction is executed.
The embodiment of the present disclosure further provides a storage medium, which stores executable instructions, and when the executable instructions are executed, the storage medium is used for implementing the segmented media file generation method provided by the embodiment of the present disclosure.
Drawings
FIG. 1 is an alternative schematic construction of a container provided in accordance with an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of an alternative package structure of an FLV file provided by an embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an audio tag file provided in an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a video tag file provided in an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a script tag file provided in an embodiment of the present disclosure;
fig. 6 is a schematic diagram of an alternative packaging structure of an FMP4 file provided by an embodiment of the disclosure;
fig. 7 is an alternative structural diagram of a segmented media file generating apparatus according to an embodiment of the disclosure;
FIG. 8 is an alternative flow chart of a segmented media file generation method provided by the embodiments of the present disclosure;
FIG. 9 is an alternative structural diagram of a segmented media file generating apparatus in accordance with the disclosed embodiments;
FIG. 10 is a schematic diagram of an alternative process for converting an FLV file into an FMP4 file for playing provided by an implementation of the present disclosure;
FIG. 11 is an alternative diagram of the present disclosure for converting an FLV file into an FMP4 file for playing;
fig. 12 is an optional flowchart illustrating that the FLV file is converted into the FMP4 file for playing in the live broadcast scenario according to the present disclosure;
fig. 13 is a schematic diagram illustrating that an FLV file is converted into an FMP4 file for playing in a live scene according to an embodiment of the present disclosure.
Detailed Description
For the purpose of making the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be described in further detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present disclosure, and all other embodiments obtained by a person of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present disclosure.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure.
Before the present disclosure is explained in further detail, terms and expressions referred to in the embodiments of the present disclosure are explained, and the terms and expressions referred to in the embodiments of the present disclosure are applied to the following explanations.
1) A media file, a file storing encoded media data (e.g., at least one of audio data and video data) in a container (Box), includes metadata, i.e., data describing the media data, and carries media information to ensure that the media data is decoded correctly.
For example, a media file formed by packaging multimedia data in a streaming media packaging format is called an FLV file, and typically, the FLV file stores therein Video data coded in an Advanced Video Coding (AVC) or MPEG-4(Part 2) specification and Audio data coded in an Advanced Audio Coding (AAC) specification, although other Coding modes of Video and Audio are not excluded.
2) A container (Box), also called a Box, an object-oriented component defined by a unique type identifier and a length, see fig. 1, is an optional structural diagram of the container provided by the embodiment of the present disclosure, and includes a container Header (Box Header) and container Data (Box Data), which are filled with binary Data to express various information.
The container header includes a size (size) that specifies the size of the storage space occupied by the container (also referred to herein as capacity or length), and a type (type) that specifies the type of container.
3) The metadata container, the container used for storing the metadata (i.e. the data describing the multimedia data stored in the media data container) in the media file, the metadata container in the FLV file is Script Tag, the information expressed by the binary data stored in the Script Tag is called the media information, and the media information contains the playing information about the video and/or audio, such as duration, creation time, modification time, etc.
4) A media data container, a container for storing multimedia data in a media File, for example, a media data container in an FLV File is a Tag (Tag), see fig. 2, which is an optional packaging structure diagram of the FLV File provided in the embodiment of the present disclosure, the FLV File is composed of an FLV Header File Header and a File Body (File Body), where the File Body is composed of one Tag and a Tag Size pair, and the Tag may be divided into three types: the Audio Tag, Video Tag, and Script Tag respectively represent an Audio stream, a Video stream, and a Script stream (such as keyword or file information), and each Tag can only contain one of the above three types of data, which will be described below.
The 1 st byte from the beginning of the Audio Tag file (Audio Tag) contains parameter information of Audio data, and the Audio Tag file is configured as Audio stream data from the 2 nd byte, referring to fig. 3, which is a schematic structural diagram of the Audio Tag file provided in the embodiment of the present disclosure, a numerical value of the first 4 bits of the 1 st byte represents an Audio coding type, a numerical value of the 5 th to 6 th bits represents an Audio sampling rate, a 7 th bit represents an Audio sampling precision, and an 8 th bit represents an Audio type.
The first 1 byte of the Video Tag file (Video Tag) contains parameter information of Video data, and the 2 nd byte is Video stream data, see fig. 4, which is a schematic structural diagram of the Video Tag file provided in the embodiment of the present disclosure, where a value of the first 4 bits of the 1 st byte represents a frame type, and a value of the last 4 bits of the 1 st byte represents a Video coding type.
Script Tag files (Script tags), which are also commonly referred to as Metadata tags, put some Metadata information about the FLV video and audio, such as: duration, width, height, etc. Generally, this type of Tag will appear as the first Tag following the File Header, and only one Tag, see fig. 5, is a schematic structural diagram of the script Tag File provided in the embodiment of the present disclosure.
5) And segmenting the media files, wherein the media files are divided into subfiles, and each segmented media file can be independently decoded.
Taking an FLV file as an example, media data in the FLV file is divided according to key frames, the divided media data and corresponding metadata are packaged to form a segmented MP4(FMP4, segmented MP4) file, and the metadata in each FMP4 file can ensure that the media data is correctly decoded.
For example, when the FLV file shown in fig. 2 is converted into multiple FMP4 files, referring to fig. 6, which is an optional packaging structure diagram of the FMP4 file provided in the embodiment of the present disclosure, one FLV file may be converted into multiple FMP4 files, and each FMP4 file includes three basic containers: moov containers, moof containers, and mdat containers.
The moov container includes MP4 file level metadata describing all media data in the MP4 file from which the FMP4 file is derived, such as the duration, creation time, and modification time of the MP4 file.
The moof container stores segment-level metadata describing media data packaged in the FMP4 file where it is located, ensuring that the media data in the FMP4 can be decoded.
1 moof container and 1 mdat container constitute 1 segment of the segment MP4 file, and 1 or more such segments may be included in the 1 segment MP4 file, and the metadata encapsulated in each segment ensures that the media data encapsulated in the segment can be independently decoded.
6) Media resource Extensions (MSE) interface, player-oriented interface implemented in a web page, interpreted by the browser's interpreter during loading in the web page, implemented by executing a front-end programming language (e.g., JavaScript), provides the player with the functionality to invoke the play Media stream of hypertext markup language (H TML) Media elements (Media elements), e.g., implement the play functionality of video/audio using video Element < video > and audio Element < audio >.
7) The streaming media format encapsulates media data into a media file of the streaming media, and the media file can be decoded and played without complete downloading and extra transcoding, namely, native support is provided for an encapsulation technology of downloading and playing simultaneously. A typical file in streaming media format includes: TS media file fragments based on HTTP Live Streaming (HLS, H TTP Live Streaming) technology, FLV (flash video) files and the like.
First, a segmented media file generating apparatus in a media file playing process for implementing the embodiment of the present disclosure is described, where the segmented media file generating apparatus in the media file playing process may be provided as hardware, software, or a combination of hardware and software.
The following describes implementation of software and hardware combination of a segmented media file generation apparatus in a media file playing process, referring to fig. 7, fig. 7 is a schematic diagram of an optional component structure of the segmented media file generation apparatus in the media file playing process according to an embodiment of the present disclosure, and the segmented media file generation apparatus in the media file playing process according to the embodiment of the present disclosure may be implemented in various forms, for example: the method is implemented by a terminal such as a smart phone, a tablet computer and a desktop computer independently or cooperatively by the terminal and a server. In the following, a hardware structure of a segmented media file generating apparatus in a media file playing process according to an embodiment of the present disclosure is described in detail, it can be understood that fig. 7 only shows an exemplary structure of the segmented media file generating apparatus in the media file playing process, and not a whole structure, and a part of the structure or a whole structure shown in fig. 7 may be implemented as needed.
The device 700 for generating a segmented media file in a media file playing process provided by the embodiment of the present disclosure includes: at least one processor 701, memory 702, user interface 703, and at least one network interface 704. The various components of the segmented media file generation apparatus 700 during playback of a media file are coupled together by a bus system 705. It will be appreciated that the bus system 705 is used to enable communications among the components. The bus system 705 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various busses are labeled in figure 7 as the bus system 705.
The user interface 703 may include, among other things, a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, or a touch screen.
It will be appreciated that the memory 702 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory.
The memory 702 in the disclosed embodiment is used to store various types of data to support the operation of the segmented media file generating apparatus 700 during the playing of a media file. Examples of such data include: any executable instructions for operating on the segmented media file generation apparatus 700 during the playing of a media file, such as the executable instructions 7021, a program implementing the segmented media file generation method during the playing of a media file according to the embodiments of the present disclosure may be included in the executable instructions 7021.
The method for generating a segmented media file in a media file playing process disclosed by the embodiment of the disclosure can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the segmented media file generation method in the media file playing process may be implemented by instructions in the form of hardware integrated logic circuits or software in the processor 701. The Processor 701 may be a general purpose Processor, a Digital Signal Processor (DS P), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 701 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present disclosure. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present disclosure may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, the storage medium is located in the memory 702, and the processor 701 reads information in the memory 702, and completes the steps of the segmented media file generation method in the media file playing process provided by the embodiment of the present disclosure in combination with hardware thereof.
The following describes a pure hardware implementation of the segmented media file generation apparatus during playing a media file, and the segmented media file generation apparatus during playing a media file according to the embodiment of the present disclosure may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processing (DSPs), Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components, and is used to implement the method for generating a segmented media file during playing a media file according to the embodiment of the present disclosure.
The following describes a pure software implementation of the segmented media file generation apparatus in the media file playing process, and the segmented media file generation apparatus in the media file playing process according to the embodiment of the present disclosure may be an application program or a plug-in, or may be implemented in a manner of combining the application program and the plug-in.
As an example, the application program may be a client dedicated to playing the media file, or may be a client that takes the media file playing function as an optional function, and is implemented by installing a corresponding plug-in.
As an example, the plug-in may be implemented as a function upgrade installation package of an application program, superimposing the function of media file playing in a specific application program; or elements in the webpage played by the media are realized by adopting the front-end language, and the function of playing the media file in the webpage is realized by being directly interpreted and executed by the webpage.
Next, taking a player embedded in a webpage, for example, the player uses a hypertext Markup Language (HTML) 5 media element of the webpage to play a media file, a method for generating a segmented media file in a playing process of the media file provided by the embodiment of the present disclosure is described, where the webpage in the embodiment of the present disclosure may be a webpage of a browser, or may be a webpage of an Application (APP) embedded in a browser kernel, and the webpage implements a player example by parsing and executing js (javascript) code of the player.
Fig. 8 is a schematic diagram of an optional flow of a segmented media file generation method provided in the embodiment of the present disclosure, and as shown in fig. 8, an optional flow of a segmented media file generation method provided in the embodiment of the present disclosure includes the following steps:
step 801: binary data included in the streaming media format file is acquired.
In an embodiment of the present disclosure, the obtaining binary data included in the streaming media format file includes: and analyzing the nesting structure of the streaming media format file, and reading out corresponding binary data according to the nesting structure. Through the technical scheme shown in the embodiment, the streaming media format with the nested structure format can be disassembled to obtain the binary data in the nested structure.
Step 802: the acquired binary data is buffered in a buffer.
Step 803: and extracting corresponding binary data from the buffer by a parser for parsing.
In an embodiment of the present disclosure, the extracting, by a parser, the corresponding binary data from the buffer for parsing includes:
extracting binary data from the buffer correspondingly in a parallel mode through resolvers of different types for resolution; the different types of resolvers include at least one of: a metadata information parser, an audio parser, and a video parser. By the technical scheme shown in the embodiment, the resolvers of different types can be used for extracting the binary data from the buffer at the same time, and the defect of slow extraction of the binary data caused by serial extraction is overcome.
In an embodiment of the present disclosure, the extracting, by a parser, the corresponding binary data from the buffer for parsing includes:
and respectively coupling the unified cache structures through resolvers of different types, and extracting binary data corresponding to the resolver types from the cache through the resolvers of different types by using the unified cache structures for resolution. By the technical scheme shown in the embodiment, the resolvers of different types can be used for extracting the binary data from the buffer at the same time, and the defect of slow extraction of the binary data caused by serial extraction is overcome.
Step 804: and caching the analysis result of the binary data into the buffer.
In an embodiment of the present disclosure, the caching the parsing result of the binary data into the buffer includes:
utilizing the uniform cache structure through resolvers of different types to cache the resolution results of the binary data corresponding to the resolvers into the cache; and sharing the analysis results of the respective binary data by the different types of analyzers and by using the uniform cache structure. Through the technical scheme shown in this embodiment, the parsing process of some parsers needs to call the parsing results of other parsers on corresponding binary data, and the set unified cache structure can be used to realize that all parsers share their respective parsing results.
In an embodiment of the present disclosure, when the parser needs to parse results based on other types of parsers to implement parsing of corresponding binary data, the parsing result of the corresponding binary data is obtained through the unified cache structure.
In an embodiment of the present disclosure, before obtaining binary data included in the streaming media format file, the method further includes: setting a corresponding type of resolver according to the binary data type included in the acquired streaming media format file; initializing the set resolver of the corresponding type and a buffer corresponding to the resolver. Through the technical scheme shown in the embodiment, the type of the parser can be flexibly set to adapt to different use environments.
Step 805: and coding the analysis result of the binary data in the buffer by an encoder to form a corresponding segmented media file.
In an embodiment of the present disclosure, after the segmented media file is formed, the segmented media file may be further sent to a media element of the web page for playing through a media source extension interface of the web page.
Through the steps 801 and 805 shown in the embodiment of the present disclosure, binary data can be obtained and analyzed from a streaming media file, the analyzed binary data is cached in the buffer, and the result of analyzing the binary data in the buffer is encoded by the encoder to form a complete segmented media file. The speed of analyzing the binary data is effectively improved, and the delay of the binary data analyzing stage in the process of generating the stream segmented media file into the segmented media file is avoided.
Fig. 9 is a schematic diagram of an optional structure of a segmented media file generating apparatus in the embodiment of the present disclosure, and as shown in fig. 9, an optional structure of a segmented media file generating apparatus 900 in the embodiment of the present disclosure includes:
a data obtaining module 901, configured to obtain binary data included in the streaming media format file;
the data obtaining module 901 is configured to cache the obtained binary data in a buffer;
in an embodiment of the present disclosure, the data obtaining module 901 is configured to parse a nesting structure of the file in the streaming media format, and read out corresponding binary data according to the nesting structure.
And the data analysis module 902 is configured to extract corresponding binary data from the buffer for analysis through the analyzer, and cache an analysis result of the binary data in the buffer.
In an embodiment of the present disclosure, the data parsing module 602 is configured to extract binary data from the buffer in a parallel manner through different types of parsers for parsing; the different types of resolvers include at least one of: a metadata information analyzer, an audio analyzer and a video analyzer. By the technical scheme shown in the embodiment, the resolvers of different types can extract binary data from the buffer at the same time, and the defect of slow extraction of the binary data caused by serial extraction is overcome.
In an embodiment of the present disclosure, the data parsing module 902 is configured to couple uniform cache structures through different types of parsers, respectively; the data parsing module 902 is configured to extract binary data corresponding to the type of the parser from the cache through the different types of parsers by using the unified cache structure, and parse the binary data. By the technical scheme shown in the embodiment, the resolvers of different types can be used for extracting the binary data from the buffer at the same time, and the defect of slow extraction of the binary data caused by serial extraction is overcome.
In an embodiment of the present disclosure, the data parsing module 902 is configured to cache, by using the unified cache structure through different types of parsers, parsing results of binary data corresponding to each parser in the cache; the data parsing module 902 is configured to share, through the different types of parsers, the parsing results of the respective binary data by using the unified cache structure. Through the technical scheme shown in this embodiment, the parsing process of some parsers needs to call the parsing results of other parsers on corresponding binary data, and the set unified cache structure can be used to realize that all parsers share their respective parsing results.
In an embodiment of the present disclosure, the data parsing module 902 is configured to obtain a parsing result of corresponding binary data through the unified cache structure.
In one embodiment of the present disclosure, the apparatus further comprises:
an initialization module (not shown in the figure) for setting a parser of a corresponding type according to the binary data type included in the acquired streaming media format file; the initialization module is used for initializing the set resolver of the corresponding type and the buffer corresponding to the resolver. Through the technical scheme shown in the embodiment, the type of the parser can be flexibly set to adapt to different use environments.
The data parsing module 902 is configured to encode, by an encoder, a parsing result of the binary data in the buffer to form a corresponding segmented media file.
In one embodiment of the present disclosure, the apparatus further comprises:
and a data transceiver module (not shown in the figure) configured to send the segmented media file to a media element of the web page for playing through a media source extension interface of the web page.
Fig. 10 is a schematic diagram of an optional process for converting an FLV file into an FMP4 file for playing, which is provided in the present disclosure, and as shown in fig. 10, an optional process for converting an FLV file into an FMP4 file for playing, which is provided in the present disclosure, includes the following steps:
step 1001: and acquiring the FLV file from the static resource of the server.
In one embodiment of the present disclosure, the FLV file may be acquired from a static resource of the server according to the request information.
Step 1002: and processing the acquired FLV file through a Tag wrapper to acquire different Tag files.
In one embodiment of the present disclosure, the processing, by the Tag packager, the acquired FLV file includes: processing different types of information of the acquired FLV file through a Tag wrapper, wherein the different types of information comprise at least one of the following information:
header metadata information, video header information, audio header information, video data, audio data.
The obtained different tag files comprise: script tags, video tags, audio tags.
Step 1003: and storing the binary data in the FLV file processed by the Tag wrapper in a cache center.
The cache center is composed of a plurality of caches, and the analysis result of the binary data can be cached in the caches.
Step 1004: and different types of decoders extract corresponding binary data from the cache center for parsing.
Wherein the different types of decoders include: a Meta information decoder, an audio decoder, and a video decoder.
Step 1005: and caching the analysis result of the binary data in a cache center.
Step 1006: the encoder obtains the analysis result of the binary data from the cache center.
Step 1007: the encoder encodes the received parsed binary data to form an FMP4 file.
In one embodiment of the present disclosure, the encoder may obtain a video tag and an audio tag in the parsed binary data.
Step 1008: the encoder pushes the formed FMP4 file to the control hub to implement control of the FMP4 file playing process.
Step 1009: and sending the segmented media file to a media element of the webpage for playing through a media source expansion interface of the webpage.
Fig. 11 is an optional schematic diagram of converting an FLV file into an FMP4 file for playing according to an implementation of the present disclosure, as shown in fig. 11, a player acquires media data of the FLV file from a server, encapsulates the media data into tag files (script tag, video tag, audio tag) of different types according to data types (script data, video data, audio data), buffers the encapsulated tag files into a buffer, invokes a parser of a corresponding type, i.e., a Meta information decoder, an audio decoder, and a video decoder shown in fig. 11, parses the cached tag files, and buffers parsing results into the buffer, invokes an encoder to acquire data parsed by a sub-parser from the buffer, encodes the data to obtain an FMP4 file, and adds the encoded FMP4 file to a media source object in a media resource extension interface, and calling a media resource expansion interface to create a virtual address corresponding to the media source object, and transmitting the virtual address to a media element (Video tag) of the webpage for Video playing. For example, the player executes the create Object URL method encapsulated in the media source extension interface, and creates a virtual address, i.e. a virtual Uniform Resource Locator (URL), of the corresponding media source Object, in which the Blob type segmented media file is encapsulated.
Fig. 12 is a schematic view of an optional process for converting an FLV file into an FMP4 file for playing in a live broadcast scene provided by the embodiment of the present disclosure, and as shown in fig. 12, an optional process for converting an FLV file into an FMP4 file for playing in a live broadcast scene provided by the embodiment of the present disclosure includes:
step 1201: and the live broadcasting main broadcasting side terminal pushes the stream to the live broadcasting server, and the live broadcasting server acquires the live broadcasting video data.
In one embodiment of the present disclosure, the player may send a long connection establishment request to the live server to establish a long connection with the live server.
Step 1202: and acquiring binary data in the corresponding FLV file from the live broadcast server through a request module.
Step 1203: and the decoder acquires the binary data in the FLV file through the request module and analyzes the binary data in the FLV file through the decoder.
In one embodiment of the present disclosure, media data retrieved from a live server may be packaged as a tag file of a different data type.
In practical application, the media data in the FLV file with the target resolution requested from the server is packaged into a script Tag, a video Tag and an audio Tag.
Step 1204: and storing the binary data in the FLV file subjected to decoding processing by the decoder in a corresponding buffer.
Step 1205: the encoder calls the analysis result of the binary data stored in the buffer to encode to form an FMP4 file.
Step 1206: the control hub calls the formed FMP4 file to realize control over the playing process of the FMP4 file.
Wherein, the player can add the obtained segmented media file to the media source object in the media resource expansion interface; and transmitting a virtual address to the media element of the webpage, wherein the virtual address is used for playing the media element by taking the media source object as a data source.
Step 1207: and sending the segmented media file to a media element of the webpage for playing through a media source expansion interface of the webpage.
Fig. 13 is a schematic diagram illustrating that an FLV file is converted into an FMP4 file for playing in a live scene according to an embodiment of the present disclosure.
After media data is collected by a main broadcasting end, the media data is pushed to a live broadcasting server, long connection is established between the live broadcasting server and a request module, media data packaged in an FLV format and sent by the server is continuously obtained through the long connection, the obtained media data is packaged into different types of label files (an audio label file, a video label file and a script label file), a controller (a control center) calls decoders (an audio decoder, a video decoder and a script decoder) of corresponding types to analyze the obtained label files, an analysis result is cached in a cache, the controller calls an encoder to encode the media data in the cache to obtain a segmented media file (FMP4 file), the constructed segmented media file is added to a media source object in a media resource expansion interface, a virtual address corresponding to the media source object is created through MSE, and sending the virtual address to a Video element of the browser for playing.
It should be noted that the JS implementation of the player includes the functions of the request module, decoder, encoder, and controller in fig. 13.
The above description is only exemplary of the present disclosure and should not be taken as limiting the scope of the present disclosure, which is intended to cover any variations, modifications, equivalents, and improvements included within the spirit and scope of the present disclosure.
Claims (16)
1. A method for generating a segmented media file, comprising:
acquiring binary data included in a nested structure of a streaming media format file, and caching the acquired binary data in a buffer;
extracting corresponding binary data from the buffer in a parallel mode through resolvers of different types to resolve, and buffering resolving results of the binary data into the buffer, wherein the resolvers of different types include at least one of the following: a metadata information analyzer, an audio analyzer and a video analyzer;
and coding the analysis result of the binary data in the buffer by an encoder to form a corresponding segmented media file.
2. The method of claim 1, wherein the extracting corresponding binary data from the buffer for parsing by different types of parsers in a parallel manner comprises:
and respectively coupling the unified cache structures through resolvers of different types, and extracting binary data corresponding to the resolver types from the cache in parallel through resolvers of different types by using the unified cache structures for resolution.
3. The method of claim 2, wherein the buffering the result of the parsing of the binary data into the buffer comprises:
caching the analysis results of the binary data corresponding to the analyzers into the cache by using the unified cache structure through different types of analyzers;
and sharing the analysis results of the respective binary data by the different types of analyzers and by using the uniform cache structure.
4. The method of claim 3, further comprising:
and when the resolver needs to resolve the corresponding binary data based on the resolving results of resolvers of other types, obtaining the resolving result of the corresponding binary data through the unified cache structure.
5. The method of claim 1, further comprising:
setting a corresponding type of resolver according to the binary data type included in the acquired streaming media format file;
initializing the set resolver of the corresponding type and a buffer corresponding to the resolver.
6. The method according to claim 1, wherein the obtaining binary data included in the nesting structure of the streaming media format file comprises:
and analyzing the nesting structure of the streaming media format file, and reading corresponding binary data according to the nesting structure.
7. The method of claim 1, further comprising:
and sending the segmented media file to a media element of the webpage for playing through a media source expansion interface of the webpage.
8. A segmented media file generating apparatus, the apparatus comprising:
the data acquisition module is used for acquiring binary data included in a nested structure of the streaming media format file;
the data acquisition module is used for caching the acquired binary data in a buffer;
the data analysis module is used for extracting corresponding binary data from the buffer memory in a parallel mode through different types of analyzers for analysis, and buffering the analysis result of the binary data into the buffer memory, wherein the different types of analyzers comprise at least one of the following components: a metadata information analyzer, an audio analyzer and a video analyzer;
and the data analysis module is used for coding the analysis result of the binary data in the buffer through an encoder so as to form a corresponding segmented media file.
9. The apparatus of claim 8,
the data analysis module is used for respectively coupling the uniform cache structures through different types of analyzers;
and the data analysis module is used for extracting binary data corresponding to the types of the resolvers from the buffer in parallel by using the unified buffer structure through the resolvers of different types for analysis.
10. The apparatus of claim 9,
the data analysis module is used for caching the analysis results of the binary data corresponding to the analyzers into the cache by using the unified cache structure through the analyzers of different types;
and the data analysis module is used for sharing the respective analysis results of the corresponding binary data by using the uniform cache structure through the analyzers of different types.
11. The apparatus of claim 10,
and the data analysis module is used for acquiring the analysis result of the corresponding binary data through the unified cache structure.
12. The apparatus of claim 8, further comprising:
the initialization module is used for setting a corresponding type of resolver according to the type of binary data included in the acquired streaming media format file;
the initialization module is used for initializing the set resolver of the corresponding type and the buffer corresponding to the resolver.
13. The apparatus of claim 8,
the data acquisition module is used for analyzing the nesting structure of the streaming media format file and reading corresponding binary data according to the nesting structure.
14. The apparatus of claim 8, further comprising:
and the data transceiving module is used for sending the segmented media file to the media element of the webpage for playing through a media source expansion interface of the webpage.
15. A segmented media file generating apparatus, comprising:
a memory for storing executable instructions;
a processor, configured to execute the executable instructions to implement the segmented media file generation method of any of claims 1 to 7.
16. A storage medium having stored thereon executable instructions for, when executed, implementing a segmented media file generation method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810974160.4A CN110858923B (en) | 2018-08-24 | 2018-08-24 | Method and device for generating segmented media file and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810974160.4A CN110858923B (en) | 2018-08-24 | 2018-08-24 | Method and device for generating segmented media file and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110858923A CN110858923A (en) | 2020-03-03 |
| CN110858923B true CN110858923B (en) | 2022-09-06 |
Family
ID=69635559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810974160.4A Active CN110858923B (en) | 2018-08-24 | 2018-08-24 | Method and device for generating segmented media file and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110858923B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101098483A (en) * | 2007-07-19 | 2008-01-02 | 上海交通大学 | Video cluster transcoding system using image group structure as parallel processing element |
| CN102630059A (en) * | 2012-03-23 | 2012-08-08 | 深圳创维数字技术股份有限公司 | Method and device for realizing FLV video play |
| CN103826138A (en) * | 2014-03-12 | 2014-05-28 | 赛特斯信息科技股份有限公司 | System and method for realizing cloud transcoding on various code rates and encoding formats of audio and video contents |
| CN104168439A (en) * | 2013-05-17 | 2014-11-26 | 腾讯科技（深圳）有限公司 | Video coding method and apparatus |
| CN105142037A (en) * | 2015-09-10 | 2015-12-09 | 北京暴风科技股份有限公司 | Distributed transcoded audio and video synthesis method and system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103369355A (en) * | 2012-04-10 | 2013-10-23 | 华为技术有限公司 | Online media data conversion method, video playing method and corresponding device |
| US9971686B2 (en) * | 2015-02-23 | 2018-05-15 | Intel Corporation | Vector cache line write back processors, methods, systems, and instructions |
| CN106792118A (en) * | 2016-12-20 | 2017-05-31 | 北京小米移动软件有限公司 | The player method and device of multimedia file |
| CN106961613A (en) * | 2017-03-30 | 2017-07-18 | 上海七牛信息技术有限公司 | A kind of streaming real-time transcoding order method and system |
-
2018
- 2018-08-24 CN CN201810974160.4A patent/CN110858923B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101098483A (en) * | 2007-07-19 | 2008-01-02 | 上海交通大学 | Video cluster transcoding system using image group structure as parallel processing element |
| CN102630059A (en) * | 2012-03-23 | 2012-08-08 | 深圳创维数字技术股份有限公司 | Method and device for realizing FLV video play |
| CN104168439A (en) * | 2013-05-17 | 2014-11-26 | 腾讯科技（深圳）有限公司 | Video coding method and apparatus |
| CN103826138A (en) * | 2014-03-12 | 2014-05-28 | 赛特斯信息科技股份有限公司 | System and method for realizing cloud transcoding on various code rates and encoding formats of audio and video contents |
| CN105142037A (en) * | 2015-09-10 | 2015-12-09 | 北京暴风科技股份有限公司 | Distributed transcoded audio and video synthesis method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110858923A (en) | 2020-03-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110545483B (en) | Method, device and storage medium for playing media file by switching resolution in webpage | |
| CN110545466B (en) | Webpage-based media file playing method and device and storage medium | |
| CN110545456B (en) | Synchronous playing method and device of media files and storage medium | |
| CN110545490B (en) | Media file conversion method, device and storage medium | |
| CN110545491B (en) | Network playing method, device and storage medium of media file | |
| CN110545479B (en) | Loading control method and device for media playing and storage medium | |
| US11025991B2 (en) | Webpage playing method and device and storage medium for non-streaming media file | |
| CN110858915A (en) | Resolution switching method and device in media file playing process and storage medium | |
| CN110858919A (en) | Data processing method and device in media file playing process and storage medium | |
| CN110545448B (en) | Media playing method and device based on data encryption and storage medium | |
| JP2021510991A5 (en) | ||
| CN110545460B (en) | Media file preloading method and device and storage medium | |
| CN110858923B (en) | Method and device for generating segmented media file and storage medium | |
| CN110545471B (en) | Playing control method and device based on offline conversion and storage medium | |
| CN110545463B (en) | Play control method and device based on media file conversion and storage medium | |
| CN110545468B (en) | Media file playing method and device based on parameter encapsulation and storage medium | |
| CN110545461A (en) | Resolution switching method and device of media file and storage medium | |
| CN110545480A (en) | Preloading control method and device of media file and storage medium | |
| CN110858917A (en) | Control method and device for media file resolution switching and storage medium | |
| CN110545464A (en) | Media file resolution switching method and device and storage medium | |
| CN110545467A (en) | Media file loading control method and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. |