CN110545482A

CN110545482A - Continuous playing method and device during resolution switching and storage medium

Info

Publication number: CN110545482A
Application number: CN201810531125.5A
Authority: CN
Inventors: 银国徽
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Douyin Vision Co Ltd; Douyin Vision Beijing Co Ltd
Priority date: 2018-05-29
Filing date: 2018-05-29
Publication date: 2019-12-06
Anticipated expiration: 2038-05-29
Also published as: CN110545482B

Abstract

The present disclosure provides a continuous playing method during resolution switching, which includes: when the player plays the segmented media file in the embedded webpage, detecting a real-time playing point for playing the segmented media file with the original resolution when a resolution switching event is received; acquiring a segmented media file for continuing the real-time playing point, wherein the segmented media file for continuing the real-time playing point is formed by a media file based on a target resolution; and loading the segmented media file with the target resolution through the player so as to continue playing the segmented media file with the original resolution in the webpage. The present disclosure also provides a continuous playing device and a storage medium during resolution switching.

Description

Continuous playing method and device during resolution switching and storage medium

Technical Field

The present disclosure relates to network media playing technologies, and in particular, to a method and an apparatus for continuous playing during resolution switching, and a storage medium.

Background

At present, the media playing by using the webpage is a commonly used video playing scheme, and the complicated operation of installing a special client can be reduced.

However, when playing a media file in a web page, if resolution switching is to be performed on the currently played media file, the media file with the corresponding resolution needs to be loaded again, which inevitably causes network delay, and further causes that a media data segment with a target resolution cannot be retrieved in time, so that the playing of the whole media file is not smooth, and a phenomenon of playing jam occurs, which seriously affects the user experience. In view of the above technical problems, no effective solution has been proposed in the related art.

Disclosure of Invention

in view of the above, it is desirable to provide a method, an apparatus and a storage medium for continuous playback during resolution switching, which are at least used to improve the efficiency of retrieving media data segments with target resolution from a web page.

in order to achieve the above purpose, the technical solution of the embodiment of the present disclosure is implemented as follows:

In a first aspect, an embodiment of the present disclosure provides a method for continuous playback during resolution switching, where the method includes:

When the player plays the segmented media file in the embedded webpage, detecting a real-time playing point for playing the segmented media file with the original resolution when a resolution switching event is received;

Acquiring a segmented media file for continuing the real-time playing point, wherein the segmented media file for continuing the real-time playing point is formed by a media file based on a target resolution;

And loading the segmented media file with the target resolution through the player so as to continue playing the segmented media file with the original resolution in the webpage.

In a second aspect, an embodiment of the present disclosure further provides a continuous playback device during resolution switching, where the device includes: the device comprises a detection module, an acquisition module and a loading module; wherein the content of the first and second substances,

The detection module is used for detecting a real-time playing point of the segmented media file with the original resolution when a resolution switching event is received in the process that the player plays the segmented media file on the embedded webpage;

The acquisition module is used for acquiring a segmented media file for continuing the real-time playing point, wherein the segmented media file for continuing the real-time playing point is formed by a media file based on a target resolution;

The loading module is configured to load the segmented media file with the target resolution through the player so as to continue playing the segmented media file with the original resolution in the webpage.

In a third aspect, an embodiment of the present disclosure further provides a continuous playing device during resolution switching, including:

A memory for storing executable instructions;

And the processor is used for realizing the continuous playing method during resolution switching provided by the embodiment of the disclosure when the executable instruction is executed.

In a fourth aspect, an embodiment of the present disclosure further provides a storage medium, where executable instructions are stored, and when the executable instructions are executed, the storage medium is configured to implement the method for continuous playing during resolution switching provided by the embodiment of the present disclosure.

According to the method, the device and the storage medium for playing continuously during resolution switching, the player plays the media file in the embedded webpage in the form of the segmented media file, the segmented media file for continuing the target resolution of the real-time playing point is obtained according to the resolution switching event, and the segmented media file continuing the original resolution is played in the webpage.

drawings

FIG. 1 is a schematic view of an alternative construction of a container provided in accordance with an embodiment of the present disclosure;

Fig. 2 is a schematic diagram of an alternative package structure of an MP4 file according to an embodiment of the disclosure;

Fig. 3 is a schematic structural diagram of a media data container storing media data in a media file according to an embodiment of the present disclosure;

Fig. 4 is a schematic diagram of an alternative package structure of an FMP4 file provided by an embodiment of the present disclosure;

Fig. 5 is a schematic diagram of an alternative architecture of a media playing control system according to an embodiment of the disclosure;

fig. 6 is a schematic diagram of an alternative hardware structure of a continuous playing device during resolution switching according to an embodiment of the present disclosure;

Fig. 7 is an optional schematic composition diagram of a continuous playing device during resolution switching according to an embodiment of the present disclosure;

Fig. 8 is a schematic diagram of another optional component of the continuous playback device during resolution switching according to the embodiment of the present disclosure;

Fig. 9 is a schematic diagram of another optional component of the continuous playback device during resolution switching according to the embodiment of the present disclosure;

Fig. 10 is a schematic view of an alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure;

Fig. 11 is a schematic flowchart of a process that a player sends a target resolution segmented media file to a media element of a web page through a media resource extension interface of the web page for decoding and playing according to the embodiment of the present disclosure;

fig. 12 is an alternative schematic diagram of a player playing a target resolution segmented media file through a media resource extension interface of a web page according to an embodiment of the present disclosure;

FIG. 13 is a schematic diagram illustrating an alternative implementation flow of packaging a target resolution segmented media file according to an embodiment of the present disclosure;

FIG. 14 is a schematic diagram illustrating an alternative process for parsing media information from a metadata container according to an embodiment of the present disclosure;

Fig. 15 is a schematic view of another alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure;

Fig. 16 is a schematic view of another alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure;

Fig. 17 is a schematic diagram of another alternative implementation flow of the continuous playing method during resolution switching according to the embodiment of the present disclosure.

Detailed Description

For the purpose of making the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be described in further detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present disclosure, and all other embodiments obtained by a person of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present disclosure.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure.

Before the present disclosure is explained in further detail, terms and expressions referred to in the embodiments of the present disclosure are explained, and the terms and expressions referred to in the embodiments of the present disclosure are applied to the following explanations.

1) a media file storing a file of encoded media data (e.g., at least one of audio data and video data) in a container (Box), which further includes metadata to express media information to ensure that the media data is correctly decoded.

For example, a formed media file in which media data is packaged in a Moving Picture Experts Group (MPEG) -4 packaging format is called an MP4 file. Typically, the MP4 file stores therein Video data encoded by the Advanced Video Coding (AVC, h.264) or MPEG-4(Part 2) specification and Audio data encoded by the Advanced Audio Coding (AAC) specification, although other encoding schemes for Video and Audio are not excluded.

2) A container (Box), also called a Box, an object-oriented member defined by a unique type identifier and a length, see fig. 1, fig. 1 is an alternative structural diagram of the container provided by the embodiment of the present disclosure, including a container Header (Box Header) and container Data (Box Data), which are filled with binary Data for expressing various information.

the container header includes a size (size) and a type (type), the size indicates a length occupied by the container in the media file, the type indicates a type of the container, referring to fig. 2, fig. 2 is a schematic diagram of an alternative package structure of an MP4 file provided by an embodiment of the present disclosure, and basic container types referred to in an MP4 file include a file type container (ftyp box), a metadata container (moov box), and a media data container (mdat box).

The container data portion may store specific data, where the container is referred to as a "data container," and may further encapsulate other types of containers, where the container is referred to as a "container of a container.

3) a Track (Track), also called a Stream (Stream), a time-ordered sequence of related samples (Sample) in a container of media data, a Track representing a sequence of video frames or a sequence of audio frames for media data, and possibly a subtitle Track synchronized with the sequence of video frames, a set of consecutive samples in the same Track being called a Chunk (Chunk).

4) a file type container, a container for storing the capacity (i.e. the length of occupied bytes) and the type of a file in a media file, as shown in fig. 2, the file type container is marked as "ftyp box", wherein the stored binary data describes the type and compatibility of the file according to the specified byte length.

5) a metadata container, a container in a media file for storing metadata (i.e., data describing multimedia data stored in the media data container), and information expressed by binary data stored in the metadata container in the MP4 file are referred to as media information.

As shown in fig. 2, the header of the metadata container represents the type of the container as "moov box" using binary data, and the container data part encapsulates an mvhd container for storing general information of an MP4 file, is independent of an MP4 file, and is related to the playing of an MP4 file, including a time length, a creation time, a modification time, and the like.

The media data container of the media file may include sub-containers corresponding to a plurality of tracks, such as an audio track container (audio track box) and a video track container (video track box), in which references and descriptions of media data of the corresponding tracks are included, and the necessary sub-containers include: a container (denoted tkhd box) for describing the characteristics and overall information of the track (e.g. duration, width, height), and a container (denoted mdia box) for recording media information of the track (e.g. information of media type and sample).

as for the sub-containers packaged in the mdia box, it may include: recording the relevant attributes and content of the track (denoted mdhd box), recording the playing procedure information of the media (denoted hdlr box), describing the media information of the media data in the track (denoted minf box); the minf box in turn has a sub-container (denoted as dinf box) for explaining how to locate the media information, and a sub-container (denoted as stbl box) for recording all the time information (decoding time/display time), position information, codec etc. of the samples in the track.

referring to fig. 3, fig. 3 is a schematic structural diagram of a media data container for storing media data in a media file according to an embodiment of the present disclosure, and the time, type, capacity and location of a sample in the media data container can be interpreted by using media information identified from binary data in a stbl box, and each sub-container in the stbl box is described below.

The stsd box contains a sample description (sample description) table, and there may be one or more description tables in each media file according to different coding schemes and the number of files storing data, and the description information of each sample can be found through the description tables, and the description information can ensure correct decoding of the sample, and different media types store different description information, for example, the description information is the structure of the image in the case of video media.

The stts box stores the duration information of the samples and provides a table to map time (decoding time) and the serial numbers of the samples, and the samples at any time in the media file can be located through the sttx box; the stts box also uses other tables to map the sample size and pointers, where each entry in the table provides the serial number of consecutive samples within the same time offset and the offset of the sample, and increments these offsets to build a complete time-sample mapping table, and the calculation formula is as follows:

DT(n+1)＝DT(n)+STTS(n) (1)

Where, stts (n) is the duration of the nth sample, DT (n) is the display time of the nth sample, and the arrangement of the samples is sorted according to the time sequence, so that the offset is always non-negative, and DT generally starts with 0, and takes the display time DT (i) of the ith sample as an example, the calculation formula is as follows:

DT(i)＝SUM(for j＝0to i-1of delta(j)) (2)

The sum of all offsets is the duration of the media data in the track.

The stss box records the sequence number of the key frame in the media file.

The sts box records the mapping relation between the samples and the blocks for storing the samples, the relation between the serial numbers of the samples and the serial numbers of the blocks is mapped through a table, and the blocks containing the specified samples can be found through table lookup.

The stco box defines the position of each block in the track, expressed in terms of the offset of the starting byte in the media data container, and the length (i.e., the size) relative to the starting byte.

the stsz box records the size (i.e., size) of each sample in the media file.

6) media data container, a container for storing multimedia data in a media file, for example, a media data container in an MP4 file, as shown in fig. 3, a sample is a unit stored in the media data container, and is stored in a block of the media file, and the lengths of the block and the sample may be different.

7) And segmenting the media files, wherein the media files are divided into subfiles, and each segmented media file can be independently decoded.

Taking an MP4 file as an example, media data in an MP4 file is divided according to key frames, the divided media data and corresponding metadata are packaged to form a segmented MP4(FMP4, Fragmented MP4) file, and metadata in each FMP4 file can ensure that the media data is correctly decoded.

for example, when converting an MP4 file as shown in fig. 2 into multiple FMP4 files, referring to fig. 4, fig. 4 is a schematic diagram of an optional packaging structure of FMP4 files provided in this disclosure, one MP4 file may be converted into multiple FMP4 files, and each FMP4 file includes three basic containers: moov containers, moof containers, and mdat containers.

The moov container includes MP4 file level metadata describing all media data in the MP4 file from which the FMP4 file is derived, such as the duration, creation time, and modification time of the MP4 file.

The moof container stores segment-level metadata describing media data packaged in the FMP4 file where it is located, ensuring that the media data in the FMP4 can be decoded.

The 1 moof container and the 1 mdat container constitute 1 segment of the segment MP4 file, and 1 or more such segments may be included in the 1 segment MP4 file, and the metadata encapsulated in each segment ensures that the media data encapsulated in the segment can be independently decoded.

8) Media resource Extensions (MSE) interface, player-oriented interface implemented in web pages, interpreted by the browser's interpreter during loading in the web page, implemented by executing a front-end programming language (e.g., JavaScript), provides the player with the functionality to call the play Media stream of hypertext markup language (HTML) Media elements (Media elements), for example, to implement the play functionality of video/audio using video Element < video > and audio Element < audio >.

9) The streaming media format encapsulates media data into a media file of the streaming media, and the media file can be decoded and played without complete downloading and extra transcoding, namely, native support is provided for an encapsulation technology of downloading and playing simultaneously. A typical file in streaming media format includes: TS media file fragments based on HTTP Live Streaming (HLS, HTTP Live Streaming) technology, FLV (flash video) files, and the like.

10) non-streaming media format, which is a packaging technique that packages media data into media files and the media files can be decoded and played after being completely downloaded, a typical file in non-streaming media format includes: MP4 files, Windows Media Video (WMV) files, Advanced Streaming Format (ASF) files, and the like.

it should be noted that the MP4 file does not natively support streaming media format playback, but the technical effect of playing while downloading can be achieved by filling invalid binary data into the transcoded media stream of the player after online transcoding or the missing part of the partially downloaded MP4 file (for example, in the case of full downloading of ftyp container and moov container, the missing part of the mdat container is replaced by invalid binary data).

a process of interaction among a player, a server, and a web page in a media playback control system implementing an embodiment of the present disclosure will now be described with reference to the accompanying drawings. Referring to fig. 5, fig. 5 is a schematic diagram of an alternative architecture of a media playing control system according to an embodiment of the present disclosure; the player is used for carrying out playing control on a media file played in a webpage, when a resolution switching event occurs to an original resolution media file being played, a media data segment with a target resolution is required to be retrieved, at the moment, the player firstly sends a playing request to the server to request to acquire a segmented media file with the target resolution behind a real-time playing point, when the player acquires the segmented media file with the target resolution for continuing the real-time playing point from the server, the segmented media file with the target resolution is sent to the webpage, and the segmented media file with the target resolution is played in an embedded webpage.

Based on the implementation architecture of the media playing control system shown in fig. 5, a continuous playing device for implementing resolution switching according to the embodiment of the present disclosure is first described, and the continuous playing device for resolution switching may be provided in a hardware, software, or a combination of hardware and software.

the following describes the implementation of the software and hardware combination of the continuous playback apparatus in the resolution switching. Specifically, a hardware configuration of a playback apparatus at resolution switching that realizes the embodiments of the present disclosure, which can be implemented in various types of players, such as an HTML5 (abbreviated as H5) player applied to a web page loaded by a browser, or an application program that plays a media file, will now be described with reference to the drawings.

As the following further describes the hardware structure of the continuous playing device during resolution switching according to the embodiment of the present disclosure, it can be understood that fig. 6 only shows an exemplary structure of the continuous playing device during resolution switching, and not a whole structure, and a part of the structure or a whole structure shown in fig. 6 may be implemented as needed.

referring to fig. 6, fig. 6 is a schematic diagram of an alternative hardware structure of a continuous playing device during resolution switching according to an embodiment of the present disclosure, which may be used in various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file. The continuous playing device 600 for resolution switching can be a mobile phone, a computer, a digital broadcast terminal, an information transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. with video playing function. The continuous playback apparatus 600 in resolution switching shown in fig. 6 may include: at least one processor 601, memory 602, user interface 603, and at least one network interface 604. The components of the playback apparatus 600 during the resolution switching are coupled together by a bus system 605. It will be appreciated that the bus system 605 is used to enable communications among the components. The bus system 605 may include a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 605 in fig. 6.

The user interface 603 may include, among other things, a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, or a touch screen.

It will be appreciated that the memory 602 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory.

The memory 602 in the disclosed embodiments includes, but is not limited to: the tri-state content addressable memory, static random access memory, is capable of storing media file data retrieved from the server to support the operation of the continuous playback device 600 at resolution switch. Examples of such data include: any executable instructions for operating on the continuous playback apparatus 600 at the time of resolution switching, such as an executable program (application program) 6021 and an operating system 6022; various types of media file data information, etc. The operating system 6022 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 6021 may include various application programs such as a client with a media file conversion function, an application program, or the like. A program for implementing the continuous playback method in resolution switching according to the embodiment of the present disclosure may be included in the executable program 6021.

The continuous playing method during resolution switching disclosed by the embodiment of the disclosure can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip having signal processing capabilities. In the implementation process, the steps of the continuous playing method during resolution switching can be completed by an integrated logic circuit of hardware in the processor 601 or an instruction in the form of software. The Processor 601 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 601 may implement or execute the sequential playing method, steps and logic block diagram provided in the embodiments of the present disclosure during resolution switching. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the continuous playing method in resolution switching provided by the embodiment of the present disclosure can be directly implemented as the execution of a hardware decoding processor, or implemented by the combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in the memory 602, and the processor 601 reads the information in the memory 602, and completes the sequential playing method when the resolution is switched, which is provided in the embodiment of the present disclosure, in combination with the hardware thereof.

the following describes a pure hardware implementation of the continuous playback Device during resolution switching, and the continuous playback Device during resolution switching according to the embodiments of the present disclosure may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), Field Programmable Gate Arrays (FPGAs), or other electronic components, and is used to implement the continuous playback method during resolution switching according to the embodiments of the present disclosure.

the following describes a pure software implementation of the continuous playing device during resolution switching, and the continuous playing device during resolution switching according to the embodiment of the present disclosure may be implemented by using an application program or a plug-in, or by combining the two.

As an example, the application may be a client dedicated to playing media files, pre-compiled with corresponding installation packages for different hardware platforms.

As an example, the plug-in may be implemented as a function upgrade installation package of an application program, in which a function of media playback control is superimposed in the application program for media playback; or elements in a webpage of media playing are realized by adopting a front-end language, and the functions of media playing and media playing control are realized in the webpage through direct interpretation and execution by a browser.

In an exemplary embodiment, the present disclosure provides a storage medium, which may be a storage medium such as an optical disc, a flash memory, or a magnetic disc, and may be a non-transitory storage medium. As an example, the software module may be located in a storage medium, which may be the memory 602 shown in fig. 6, and the processor 601 reads information in the memory 602 and completes, in combination with hardware thereof, the continuous playing method applied to the media playing control system at resolution switching provided by the embodiment of the present disclosure, where the storage medium has stored thereon executable instructions that, when executed, implement the continuous playing method at resolution switching provided by the embodiment of the present disclosure.

As an example of a hardware implementation or a software implementation of the continuous playing device during resolution switching, the continuous playing device during resolution switching may be provided as a series of modules having a coupling relationship in a signal/information/data layer, which will be described below with reference to fig. 7, fig. 7 is an optional schematic composition diagram of the continuous playing device during resolution switching according to the embodiment of the present disclosure, and shows a series of modules included in the continuous playing device during resolution switching, but the module structure of the continuous playing device during resolution switching is not limited to that shown in fig. 7, and for example, the modules therein may be further split or combined according to different functions implemented.

The following describes a pure hardware implementation of the continuous playing device during resolution switching, where the continuous playing device during resolution switching may be used to run various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file, and fig. 7 is an optional schematic composition diagram of the continuous playing device during resolution switching according to the embodiment of the present disclosure; as shown in fig. 7, the continuous playing device during resolution switching includes a detecting module 701, an obtaining module 702, and a loading module 703. Each module is explained in detail below.

The detecting module 701 is configured to detect a real-time playing point of the segmented media file with the original resolution when the player receives a resolution switching event in a process of playing the segmented media file on the embedded web page.

An obtaining module 702, configured to obtain a segmented media file for continuing the live playing point, where the segmented media file for continuing the live playing point is formed by a media file based on a target resolution.

A loading module 703, configured to load, by the player, the segmented media file with the target resolution so as to continue playing the segmented media file with the original resolution in the web page.

In the embodiment of the present disclosure, for the obtaining module 702 to obtain the segmented media file for continuing the live play point, the following manner may be adopted to implement: first, metadata in a media file of a target resolution is requested from a server; then, according to the media information identified from the metadata, determining two key frames for continuing the real-time playing point to request the media data between the two key frames; finally, the segmented media file of the corresponding target resolution is formed according to the requested media data and the corresponding metadata package.

In an optional example of the present disclosure, the loading module 703 is further configured to, during the process of playing the segmented media file with the original resolution, pre-load, by the player, at least one segmented media file with the original resolution after the real-time playing point.

the obtaining module 702 is further configured to, when the segmented media file with the target resolution is obtained and the next segmented media file with the original resolution is already being played, continue to request to obtain the next segmented media file with the target resolution for continuing the live play point until the next segmented media file with the target resolution is retrieved before the playing of the next segmented media file with the original resolution is finished.

In another optional example of the present disclosure, fig. 8 is a schematic diagram illustrating another optional component of a continuous playing device during resolution switching according to an embodiment of the present disclosure; as shown in fig. 8, the loading module 703 in the continuous playing device during resolution switching is further configured to pre-load, by the player, at least one segmented media file with the original resolution after the real-time playing point in the process of playing the segmented media file with the original resolution.

the device also includes: a playing module 704, configured to continue to play the next segmented media file with the original resolution that has been preloaded on the webpage through the player when the real-time playing point of the segmented media file with the original resolution is the end time of the currently played segmented media file with the original resolution.

In another optional example of the present disclosure, fig. 9 is another optional schematic composition diagram of a continuous playing device during resolution switching according to an embodiment of the present disclosure, and as shown in fig. 9, the continuous playing device during resolution switching further includes: the storing module 705 is configured to, before the detecting module 701 detects a real-time playing point of the segmented media file with the original resolution when the resolution switching event is received, store the received playing request in a buffer queue for queuing according to a receiving sequence of the playing request when the playing request is received by the player.

An allocating module 706, configured to, after the storing module stores the received play request in a buffer queue for queuing, allocate a connection to the play request in the buffer queue according to an upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order; wherein the connection is for the player to request a segmented media file to be played.

The continuous playing device during resolution switching further comprises: a management module 707, configured to, when the obtaining module 702 obtains the segmented media file of the target resolution that is continuous to the real-time playing point, clear the playing request of the original resolution in the buffer queue, and cancel the connection already allocated to the playing request of the original resolution.

In another optional example of the present disclosure, the management module 707 is further configured to empty the pre-cached segment media file at the original resolution succeeding the live play point when the acquiring module 702 acquires the segment media file at the target resolution succeeding the live play point.

so far, the following description has been given of the continuous playing device during resolution switching according to its function, and a processing procedure of the continuous playing method during resolution switching according to the embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings based on the optional hardware structure diagram and the composition structure diagram of the continuous playing device during resolution switching.

Fig. 10 is a schematic diagram of an optional implementation flow of the continuous playing method during resolution switching according to the embodiment of the present disclosure, and a terminal device implementing the embodiment of the present disclosure may implement the continuous playing method during resolution switching by operating various types of players, and as for the terminal device, the terminal device may be various terminal devices such as a desktop computer and a notebook computer.

As an example, the player is an H5 player embedded in a web page loaded by a browser (or any application embedded in the browser), and is implemented by using a front-end-oriented language (e.g., JavaScript). Preloading means that the player adds the media file segment to the cache of the media source of the MSE, and the media element of the webpage is played according to the virtual address (the media source is used as a data source).

As another example, a player is an application program for playing a media file, installed into a terminal device by running a pre-compiled and encapsulated installation package program in the terminal device. Player loading refers to loading into the cache of the player.

As shown in fig. 10, a flow of implementing the continuous playing method in resolution switching in the embodiment of the present disclosure will be described with reference to the steps shown in fig. 10.

Step 1001: when the player plays the segmented media file in the embedded webpage, the player detects a real-time playing point for playing the segmented media file with the original resolution when receiving a resolution switching event.

Step 1002: and the player acquires a segmented media file for continuing the real-time playing point, wherein the segmented media file for continuing the real-time playing point is formed by a media file based on a target resolution.

Here, the player is embedded in the web page to play the segmented media files (the segmented media files suitable for the original resolution and the segmented media files suitable for the target resolution), that is, the player is displayed in the playing window of the web page in an embedded manner. In practical applications, when the player plays a plurality of media files in a web page, the media files can be played in parallel through a plurality of concurrent playing windows in the web page.

in the embodiment of the present disclosure, the real-time playing point is continuously updated according to the playing progress of the segmented media file with the original resolution. Here, the play point may refer to a time reached by the skip operation, for example, the original play point is 20% of the play progress, and the play point after the skip operation is 30% of the play progress; of course, the playing point may also refer to the time when the continuous playing mode arrives, for example, the playing point corresponding to the playing time from the 30 th minute to the 40 th minute.

In general, there may be multiple segmented media files for an entire media file, including the following two cases:

Case 1) the media file played in the web page may be a file supporting Streaming media playing itself, and is a set of a series of segmented media files, for example, a Streaming media file based on hypertext transfer protocol Live Streaming (HLS, HTTP Live Streaming), and the media file is substantially a set of a series of continuous TS files (i.e., segmented media files), and the TS files can be independently decoded and played, so that the media file can natively support downloading and playing at the same time. The number of one request for the segmented media file of the acquired target resolution, that is, the TS file whose decoding time is after the real-time playing point, may be one or more.

Case 2) the media file played in the web page is itself a non-streaming media file that does not support streaming media, such as MP4 file, the segmented media file is a segmented MP4 file (i.e. FMP4 file), the player extracts media data and metadata from the MP4 file, and after appropriate processing (calculating new metadata according to the packaging structure of FMP 4) and filling up according to the packaging structure of FMP4, the file can be decoded and played independently.

For the above case 1), since the target resolution segmented media file is a streaming media file, the player plays the target resolution segmented media file in the embedded webpage based on the webpage receiving the play request sent by the player, that is, directly obtaining the target resolution segmented media file to be played through the play request; for the above case 2), since the target resolution media file is a non-streaming media file, the player needs to encapsulate the non-streaming media file and the corresponding metadata after requesting the target resolution media data to form the corresponding target resolution segmented media file, instead of directly requesting to obtain the target resolution segmented media file.

In one embodiment, for the constructed target resolution segmented media file, the following describes an implementation manner of playing the target resolution segmented media file by a player through a media resource extension interface of a web page in conjunction with the drawings.

referring to fig. 11, fig. 11 is a schematic flowchart of a process that a player sends a target resolution segmented media file to a media element of a web page through a media resource extension interface of the web page for decoding and playing, which will be described with reference to the steps shown in fig. 11.

step 1101: the player adds the target resolution segmented media file to the media source object in the media asset extension interface.

referring to fig. 12, fig. 12 is an optional schematic diagram of a player playing a target resolution segmented Media file through a Media resource extension interface of a web page according to an embodiment of the present disclosure, where when the player receives a play event of the Media file with an original resolution at a play window (corresponding to a play window) in the web page, the player creates a Media Source object by executing a MediaSource method encapsulated in the Media resource extension interface; executing an addSource buffer method packaged in a media resource expansion interface to create a buffer of a MediaSource object, namely a Source buffer (Source buffer) object, wherein one MediaSource object has one or more Source buffer objects, and each Source buffer object can be used for corresponding to a playing window in a webpage and is used for receiving a target resolution segmented media file to be played in the window.

In the playing process of the target resolution media file, a Parser (Parser) in the player continuously constructs a new target resolution segmented media file by parsing newly acquired media data, and adds the target resolution segmented media file to a source buffer object of the same MediaSource object by executing an appndbuffer method of the source buffer object.

Step 1102: and the player calls the media resource expansion interface to create a virtual address corresponding to the media source object.

In addition, the player sets the MediaSource object as the source (src) attribute of the virtual URL, i.e., binds the virtual URL with a media element in the web page, such as a video/audio element, which is also referred to as associating the media source object to the media element in the web page.

step 1103: the player transmits a virtual address to the media element of the webpage, and the virtual address is used for playing the media element by taking the media source object as a data source.

In one embodiment, an implementation of allocating connections for play requests in a buffer queue is described, where when a play request from a web page is received by a player, the received play requests are stored in the buffer queue for queuing according to a receiving sequence of the play requests; and distributing connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence of the playing requests.

here, the buffer queue exists independently of the web page loaded by the embedded browser, and the buffer queue has an upper limit on the number of concurrent connections that can be used. When at least two playing requests initiated by the embedded player in the webpage exist, the received playing requests are stored in the buffer queue for queuing waiting according to the receiving sequence of the at least two playing requests, corresponding connection is allocated to the playing requests in a mode of queue entering first and waiting for allocation first, and the connection is used for enabling the player to request the segmented media files to be played.

Illustratively, the play request may be a request based on a hypertext Transfer Protocol (HTTP) initiated by the player through a web page, or may be a request based on a hypertext Transfer Protocol over Secure Socket Layer (HTTPs), which may prevent data from being intercepted and cracked by a third party during transmission through encryption transmission.

It should be noted that the number of the play request allocation connections in the buffer queue does not exceed the upper limit of the number of concurrent connections, the number of concurrent connections that can be used by the buffer queue may be used to indicate the number of connections that can be simultaneously used by the buffer queue, and generally, when the upper limit of the data of the concurrent connections of the buffer queue is not reached, each play request can be allocated a connection to request media data from the server.

The number of concurrent connections may be determined according to a requirement of a user for performance of a service in a web page, for example, when a delay tolerance of the user to other services (such as non-video playing or web browsing) in the web page is higher, a value of the configurable number of concurrent connections is larger than a value of a configurable number of concurrent connections when a delay tolerance of some services is lower.

The following describes an implementation manner for allocating connections for the play requests in the buffer queue according to an upper limit of the number of concurrent connections that can be used by the buffer queue and a receiving order, and may be implemented in at least one of the following two manners:

Mode 1) when the connection allocated to the received play request does not reach the upper limit of the number of concurrent connections, according to the receiving sequence, allocating corresponding connections to the play requests in the buffer queue in sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

Mode 2) when the connection allocated to the received play request reaches the upper limit of the number of concurrent connections, clearing the play request for releasing the connection in the buffer queue, and allocating corresponding connection to the play request in the buffer queue according to the receiving sequence until the number of allocated connections reaches the upper limit of the number of concurrent connections.

For the above mode 1), a strategy of allocating connections first is mainly adopted, that is, corresponding connections are allocated to all the play requests queued in the buffer queue according to the receiving sequence of the received play requests.

As for the above mode 2), when the number of connections allocated to the received play request reaches the upper limit of the number of concurrent connections, the play requests without connection allocation may still be stored in the buffer queue, and for these requests, idle connections need to be waited for, so that it is to be monitored whether there is a situation that the play request releases the connection in the buffer queue in real time, and when it is monitored that there is a play request to release the connection in the buffer queue, the play request releasing the connection in the buffer queue may be cleared to obtain an idle connection.

Here, the upper limit of the number of concurrent connections may have a statically configured attribute, and the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page. For example, the upper limit of the number of concurrent connections configured statically can be configured in the player through the attribute open interface. Specifically, the upper limit of the number of concurrent connections may support static configuration by an developer of a player or a service party operating a video through an attribute open interface. The upper limit of the number of concurrent connections which can be used by the buffer queue is greater than or equal to 2 and less than or equal to the number of concurrent playing windows in the webpage.

It should be noted that the upper limit of the number of concurrent connections is adaptively configured according to the requirement of the concurrent playing window in the web page. In the embodiment of the present disclosure, the minimum value of the upper limit of the number of concurrent connections is set to 2 instead of 1, mainly because there is a situation that two playing windows play simultaneously in a web page, and compared to the situation that the minimum value of the upper limit of the number of concurrent connections is 1, the minimum value of the upper limit of the number of concurrent connections is set to 2, which can avoid the situation that a playing window played later needs to wait because there is no response (because there are only 1 connection), and thus the two playing windows have a real-time response performance.

in this disclosure, an upper limit of the number of concurrent connections that can be used by the buffer queue may be preset, where the upper limit of the number of concurrent connections may be set by a player according to a connection number configuration specification of an operating system, or may be set by the player receiving the upper limit of the number of concurrent connections of a browser; of course, the setting may be performed autonomously by the player, for example, according to the characteristic parameters of the host device.

The following illustrates a scheme in which the player is set according to the characteristic parameters of the host device: firstly, detecting characteristic parameters of host equipment of the player; then, according to the characteristic parameters, dynamically determining the upper limit of the number of concurrent connections adapting to the performance of the host device, wherein the number of concurrent connections is smaller than the upper limit of the number of concurrent connections of the web page.

In one example, for dynamically determining the upper limit of the number of concurrent connections that adapt the performance of the host device according to the characteristic parameter, the following method may be adopted: and when the change of the characteristic parameter of the host equipment meets the change condition, determining the upper limit of the number of concurrent connections for adapting the change amplitude.

For example, when it is detected that a change in a characteristic parameter of a host device of the player has significant jitter, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adjusted down according to the jitter amplitude; when a significant improvement in the change in the characteristic parameters of the host device of the player is detected, the upper limit of the number of concurrent connections that can be used by the buffer queue can be adaptively adjusted according to the improved quantization width.

Here, for the implementation manner of lowering or raising the upper limit of the number of concurrent connections, the upper limit of the number of concurrent connections may be lowered or raised in an equal proportion, or may be lowered or raised in a non-equal proportion, and the embodiment of the present disclosure is not specifically limited herein.

The following description will take an example in which the upper limit of the number of concurrent connections is adjusted in a stepwise manner by proportionally lowering or raising the upper limit of the number of concurrent connections. Assuming that the initial value of the upper limit of the number of concurrent connections that can be used by the buffer queue is 8, the jitter amplitude of the characteristic parameter is set to change once, and the initial value of the upper limit of the number of concurrent connections is reduced by a value in equal proportion, when it is detected that the change of the characteristic parameter of the host device of the player has obvious jitter, and the jitter amplitude is changed from 5 to 3, it can be seen that the jitter amplitude of the characteristic parameter is changed twice, and accordingly, the initial value of the upper limit of the number of concurrent connections can be reduced from 8 to 6. The improved quantization amplitude of the characteristic parameter is preset to be increased by a numerical value in an equal proportion every time the improved quantization amplitude of the characteristic parameter changes once, when the change of the characteristic parameter of the host equipment of the player is detected to be obviously improved, the improved quantization amplitude is changed from 2 to 5, and therefore the improved quantization amplitude of the characteristic parameter is changed three times, the initial value of the upper limit of the concurrent connection number can be increased from 8 to 11.

In another example, the number of concurrent connections may be set according to network parameters of the web page: and detecting the network parameters of the webpage, and dynamically determining the upper limit of the number of concurrent connections according to the network parameters. For example, the network parameters may include the network bandwidth that the browser is able to use, and the network latency of the play request.

Here, taking the network parameter as the network bandwidth that can be used by the browser as an example, when the network bandwidth is higher, the delay for retrieving the media data is smaller, so that a smaller upper limit of the number of concurrent connections can be configured at this time when the bandwidth is lower, that is, the network bandwidth and the upper limit of the number of concurrent connections are in a negative correlation relationship, that is, the higher the network bandwidth is, the smaller the upper limit of the number of concurrent connections is; the lower the network bandwidth, the larger the upper limit on the number of concurrent connections.

It should be noted that the target resolution media file described in the embodiments of the present disclosure may be in a streaming media format or a non-streaming media format.

In one embodiment, for the case that the target resolution media file is a streaming media file, the play request is used to request the server for the target resolution segmented media file within a given time period (for continuing the real-time playing point of the player), and the media element sent to the web page through the media resource extension interface of the web page is decoded, so as to implement continuous playing of the target resolution media file.

As an example, the given period is a preload duration after the play point for preloading a portion of the media file after the play point for a smooth viewing experience. The length of the given period may be adapted by the player to network parameters or characteristic parameters of the hosting device to achieve an optimized utilization of terminal resources and/or network resources.

As an example, the given period may also be the length of at least one content unit after the playing point, wherein the content unit is divided and formed according to characters, scenes, plots and the like in the media file to represent the change of the content in the media file, so as to avoid the given period from being jumped by the user to consume unnecessary traffic to the greatest extent.

In one embodiment, for the case that the target resolution media file is a non-streaming media file, the player requests media data between two key frames from the server by determining a given time period in the target resolution media file following a real-time playing point (the time period defined between the decoding of the key frames forms the given time period or includes the given time period), so as to construct a target resolution media file capable of being independently decoded and played, and transmits the target resolution media file to a media element of a web page through a media resource extension interface of the web page for decoding, thereby implementing continuous playing of the target resolution media file.

The time corresponding to the playing point is a time measurement relative to a media time coordinate system (with the playing start time of the media file as a time origin), and the length of the given time interval is smaller than the length of the media file, for example, 5% of the predetermined proportion of the length of the media file, or a set length such as 10 minutes.

the following continues with a description of the manner in which the corresponding two key frames in the target resolution media file are determined according to a given time period.

in this embodiment of the present disclosure, for the segmented media file obtained in this step 1002 to continue the live play point, the following method may be adopted: requesting metadata in a media file at a target resolution from the server; then, according to the media information identified from the metadata, determining two key frames for continuing the real-time playing point, and requesting the media data between the two key frames from the server; finally, the segmented media file of the corresponding target resolution is formed according to the requested media data and the corresponding metadata package.

For example, when the target resolution media file is an MP4 file (file that does not support streaming media playback), the target resolution segmented media file is formed based on two key frames determined in the media file, extracting video frames between the two key frames and aligned audio frames from the target resolution media file, calculating corresponding metadata, and filling in according to the packaging format of FMP 4.

In one embodiment, an implementation manner of determining two key frames (a first key frame and a second key frame whose decoding time is after the first key frame) is described according to a case where a video frame corresponding to a playback point and a video frame corresponding to an end time of a given period are normal frames or key frames, where the playback point is a playback time that is reached by continuously playing a target resolution media file.

case 1) the video frame corresponding to the playing point is a normal frame, and since the player uses the media data between two key frames as a basic playing loading unit, the media data before the first key frame (the key frame whose decoding time is later than the key frame closest to the playing point in the key frames of the playing point) after the playing point is the loaded media data, and in order to avoid repeatedly acquiring the loaded media data, the first key frame in the two key frames of a given time period is: the first key frame in the target resolution media file whose decoding time is after the play point.

Case 2) the video frame corresponding to the playing point is a key frame, and the first key frame of the two key frames in the given period is: the corresponding key frame of the playing point, i.e. the key frame aligned with the start time of the given period.

Case 3) if the video frame corresponding to the end time of the given time period is an ordinary frame, since the player uses the media data between two key frames as a basic playing loading unit, if the key frame before the end time is used as the second key frame of the given time period, the media data between the key frame and the video frame corresponding to the end time is missed to be acquired, and when the target resolution media file is played, the media data between the key frame before the end time and the video frame corresponding to the end time cannot be played and skipped, so that in order to ensure that the video frame corresponding to the end time of the given time period can be played normally without frame skipping, the second key frame of the two key frames of the given time period is: key frames closest to the end time among the key frames whose decoding time is later than the end time of the given period.

Case 4) the video frame corresponding to the end time of the given period is a key frame, and the second key frame of the two key frames of the given period is: the second key frame, which is time-aligned with the end time of the given period, is decoded, i.e., the key frame aligned with the end time of the given period.

in the above cases 1) and 3), the key frame crossing the playing point is used as the end point of the media data in a given period, so that it can be ensured that the video frame corresponding to the playing point has enough information for correct decoding, and frame skipping due to lack of decoded data (i.e. key frame) does not occur.

In the above cases 2) and 4), for the case that the playing point aligns the key frames, the aligned key frames are directly used as the end points of the media data in the given time period, so as to reduce the situation of requesting redundant data to the maximum extent and avoid the situation that the occupation of connection and traffic causes the delay of non-media playing service in the web page.

in another embodiment, an implementation manner of determining two key frames (a first key frame and a second key frame after the first key frame at the decoding time) is described according to a case where a video frame corresponding to a play point and a video frame corresponding to the end time of a given period are normal frames or key frames, where the play point is a play time that arrives by means of jumping.

case 1) the video frame corresponding to the play point is a normal frame, and since the play point is reached by jumping, the first key frame before the play point and the media data between the play point are not loaded, and the first key frame is: in the target resolution media file, the first key frame before the playing point, that is, the key frame whose decoding time is earlier than the starting time of the given period and closest to the starting time, is searched in the time of the media data (that is, the corresponding relationship between the sequence number represented by the media information and the decoding time of the frame).

By additionally requesting the media data between the playing point and the key frame before the playing point, the normal decoding of jumping to any playing point can be ensured, and the condition that the frame jumps due to the fact that the frame can not be decoded when the playing point corresponds to the common frame is avoided.

Case 2) the video frame corresponding to the play point is a key frame, and the first key frame is: the key frame corresponding to the playing point, that is, the key frame whose decoding time searched from the time of the media data (that is, the corresponding relationship between the sequence number represented by the media information and the decoding time of the frame) is aligned with the start time of the given period.

Case 3) the video frame corresponding to the end time of the given period is a normal frame, and the second key frame is: the key frame whose decoding time is later than the end time of the given period and is closest to the end time is decoded.

Case 4) the video frame corresponding to the end time of the given period is a key frame, and the second key frame is: the key frames that are time aligned with the end time of a given period are decoded.

in the cases 2) and 4), the media data to be acquired is defined by aligning the key frames of the playing points, so that on the premise that the playing points can be correctly decoded, the situation of acquiring unnecessary media data is reduced to the greatest extent, occupation of connection and flow is reduced, and real-time performance of non-media playing services in the webpage is further ensured.

Firstly, packaging target resolution media data and metadata corresponding to the media data into a corresponding target resolution segmented media file; and then, transmitting the target resolution segmented media file to a media element of a webpage through a media resource expansion interface for playing.

In one embodiment, the player encapsulates into a corresponding target resolution segmented media file by: media data corresponding to a given time period in the target resolution media file is acquired from the server, wherein the given time period is used for continuing a playing point, the media data and metadata describing the media data are packaged according to a packaging structure of the target resolution segmented media file, so that the target resolution segmented media file which can be independently decoded by media elements of a webpage is formed, and the description is continued with reference to fig. 13.

Referring to fig. 13, fig. 13 is a schematic diagram of an alternative implementation flow of packaging a target-resolution segmented media file according to an embodiment of the present disclosure, which will be described with reference to the steps shown in fig. 13.

Step 1301: data representing the type and compatibility of the target resolution segmented media file is filled into a file type container of the target resolution segmented media file.

Step 1302: and filling metadata which represents the file level of the target resolution segmented media file into a metadata container of the target resolution segmented media file.

In one embodiment, metadata describing media data required to fill a nested structure is calculated based on the media data to be filled in the encapsulation structure of the segmented media file to the target resolution and based on the nested structure of the metadata container in the segmented media file at the target resolution.

step 1303: correspondingly filling the extracted media data and the metadata describing the media data into a media data container in a segment container of the target resolution segmented media file and a metadata container at a segment level.

In one embodiment, one or more segments (fragments) may be encapsulated in the target resolution segmented media file, and for the media data to be filled, one or more segmented media data containers (i.e., mdat box) of the target resolution segmented media file may be filled, and a segment-level metadata container (denoted as moof box) is encapsulated in each segment, wherein the filled metadata is used to describe the media data filled in the segment, so that the segments can be independently decoded.

In one embodiment of filling data into the corresponding container in steps 1301 to 1303, when the filling operation is performed, a write operation function of a class is invoked to complete writing and merging of binary data in the memory buffer of the child container, and an instance of the class is returned, where the returned instance is used to merge the child container and the child container having a nested relationship.

Before the target resolution segmented media file is packaged, metadata of media data to be filled needs to be calculated, which needs to be calculated in combination with the metadata in the target resolution segmented media file to obtain metadata at the level of the target resolution segmented media file (for example, corresponding to the metadata filled into the moov box for the FMP4 file), and metadata at the level of segments in the target resolution segmented media file (for example, corresponding to the metadata filled into the moof box for the FMP4 file).

In the following, an exemplary implementation is described that parses metadata encapsulated in a metadata container of a target resolution media file to obtain media information describing media data encapsulated in a media data container of the target resolution media file.

In one embodiment of the present disclosure, the target resolution media file is an MP4 file, the nested structure of the sub-containers in the metadata container of the target resolution media file is parsed, and binary data in each sub-container is read according to the nested structure; and analyzing the media information of the media data represented by each sub-container from the read binary data.

in one embodiment of the present disclosure, a manner is provided in which a parser is set according to a container type, and a child container in a metadata container is parsed according to the container type, so as to obtain media information, which will be described with reference to fig. 14.

Referring to fig. 14, fig. 14 is an alternative flowchart of parsing media information from a metadata container according to an embodiment of the present disclosure, which will be described with reference to the steps shown in fig. 14.

step 1401: the location of the metadata container in the target resolution media file is located.

In one embodiment, the offset and the size of the metadata container in the target resolution media file are located according to the type and length of the container identified in the read binary data by reading binary data that conforms to the container header specification from the binary data of the target resolution media file.

Step 1402: and acquiring binary data corresponding to the metadata container from the binary data of the target resolution media file according to the position of the metadata container in the target resolution media file.

The position of the metadata container in the target resolution media file is represented by an offset and a capacity, and the binary data is read from the position corresponding to the offset from the target resolution media file until the length of the read binary data conforms to the capacity of the metadata container, so that the binary data corresponding to the metadata container is read.

Step 1403: and sequentially analyzing binary data corresponding to the standard length of the container head in the binary data of the metadata container to obtain the container type of the sub-container in the metadata container and the length of the container data of the sub-container.

in one embodiment, for the case where multiple sub-containers are nested in the metadata container, the offset of each reading of binary data is the sum of the lengths of the identified sub-containers, and the length of the read binary data conforms to the canonical length of the container header, so that the type and length of the currently processed sub-container can be parsed.

Step 1404: and calling an analyzer of a type corresponding to the container type of the sub-container, and sequentially analyzing binary data corresponding to the length of the container data in the unresolved data to obtain the media information represented by the container data.

In an optional example of the present disclosure, in the process of playing a segmented media file of an original resolution in a webpage by a player, at least one segmented media file of the original resolution after the real-time playing point is pre-loaded by the player; and when the player acquires the segmented media file with the target resolution and detects that the next segmented media file with the original resolution is played, continuing to request to acquire the next segmented media file with the target resolution for continuing the real-time playing point until the next segmented media file with the target resolution is retrieved before the playing of the next segmented media file with the original resolution is finished.

In another optional example of the present disclosure, in the process of playing the segmented media file of the original resolution in the webpage, the segmented media file of at least one original resolution after the real-time playing point is pre-loaded by the player; and when the real-time playing point of the segmented media file with the original resolution is determined to be the end time of the currently played segmented media file with the original resolution, continuing to play the next segmented media file with the preloaded original resolution in the webpage through the player.

Here, the media data corresponding to the pre-loaded segmented media file is retrieved through the play request, and the media data corresponding to one segmented media file can be retrieved through each play request; that is, for the entire media file, the data retrieved each time may be encapsulated into a virtual URL object by one or more playback requests, so that the media elements of the web page may pass the virtual URL to play the segmented media file.

it should be noted that, in the embodiment of the present disclosure, the segmented media file with the original resolution and the segmented media file with the target resolution are not the same segmented media file, that is, when the player detects that the resolution switching event is received, the segmented media file for continuing the real-time playing point is formed based on the media file with the target resolution stored in the server.

Step 1003: and loading the segmented media file with the target resolution through the player so as to continue playing the segmented media file with the original resolution in the webpage.

In an embodiment of the disclosure, the method further comprises: when the segmented media file of the target resolution ratio of the continuous real-time playing point is obtained, the playing request aiming at the original resolution ratio in the buffer queue is emptied, and the connection which is already distributed aiming at the playing request aiming at the original resolution ratio is cancelled.

In another embodiment of the present disclosure, the method further comprises: and when the segmented media file of the target resolution ratio continuing to the real-time playing point is acquired, emptying the pre-cached segmented media file of the original resolution ratio continuing to the real-time playing point.

For example, the segmented media data is retrieved by the playing request (i.e. one segmented media data is retrieved by each playing request respectively here), when the occurrence of the resolution switching event is detected, assuming that the buffering time is 15s, the media data of 11s-25s has been retrieved by three playing requests when the player plays to 10s in the web page, for example, the retrieved three segmented media data are as follows: 10s-12s, 12s-18s, and 18s-25 s.

When the player determines that a resolution switching event occurs (for example, switching from the current resolution (ultra-clear) to the target resolution (high-clear)), the real-time playing point for playing the ultra-clear segmented media file when the resolution switching event is received is detected to be 11s, and for the playing request stored in the buffer queue, the following management operations (the following operations are not in order):

A) Assuming that the request of high definition data is completed within 15s during the playing of the ultraclean segmented media file, the resolution switching is requested, and it is detected that the real-time playing point of the ultraclean segmented media file has been loaded from 11s to 12s at this time, since 12s is the end time of the currently played ultraclean segmented media file, the player continues to play the webpage using the already preloaded ultraclean next segmented media file, i.e. playing the webpage using the already preloaded ultraclean 12s-18s segmented media file.

B) Assuming that a high-definition segmented media file for succeeding 11s has been acquired at the time of sending the request, the play request for super-definition data after 11s in the buffer queue is cleared, and the connection that has been allocated for the play request for super-definition is cancelled (i.e., the super-definition data after 11s is not preloaded any more).

C) Assuming that the high definition data after 11s is requested is essentially the high definition data corresponding to the key frame after 11s is requested, and the timestamp corresponding to the key frame adjacent to 11s may be exactly 11s, or may be any time instant after 11s, for example 13s, assuming that the timestamp corresponding to the key frame adjacent to 11s is between 13s-17s, and the super definition data has been played to 12s when the high definition data is retrieved, since the super definition data of 12s-18s has been played but cannot be cleared, the high definition data after 18s is continuously requested, assuming that the high definition data of 18s-25s is requested, when retrieved before 18s is played, the super definition data of 18s-25s is cleared, and played using the high definition data of 18s-25 s.

By adopting the technical scheme of the embodiment of the disclosure, the segmented media file with the original resolution is preloaded by adopting a preloading mechanism, the media file is played in the embedded webpage in the form of the segmented media file by the player, the segmented media file with the target resolution for continuing the real-time playing point is obtained according to the resolution switching event, and the segmented media file with the original resolution is played in the webpage, so that the network delay caused by the fact that the media file with the target resolution needs to be requested again is overcome, the whole media file is played smoothly, the phenomenon of playing jam is avoided, the efficiency of retrieving the media data segment with the target resolution from the webpage can be effectively improved, and the viewing experience of a user is greatly improved.

fig. 15 is a schematic view of another alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure; the continuous playing method during resolution switching can be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 15, a flow of implementing another continuous playing method during resolution switching in the embodiment of the present disclosure will be described with reference to the steps shown in fig. 15.

Step 1501: when the player plays the segmented media file in the embedded webpage, the player detects a real-time playing point for playing the segmented media file with the original resolution when receiving a resolution switching event.

in the embodiment of the present disclosure, the play point may be a time reached through the skip operation, for example, the original play point is 20% of the play progress, and the play point after the skip operation is 30% of the play progress; of course, the playing point may also be the time when the continuous playing mode arrives, for example, the playing point corresponding to the playing time from the 30 th minute to the 40 th minute.

Step 1502: in the process of playing the segmented media files with the original resolution on the webpage, at least one segmented media file with the original resolution behind the real-time playing point is preloaded through the player.

when the media data in the target resolution segmented media file is obtained through the playing request, firstly, the target resolution media data and metadata corresponding to the media data are packaged into the corresponding target resolution segmented media file; and then, transmitting the target resolution segmented media file to a media element of a webpage through a media resource expansion interface for playing.

it is noted that the media data in the target resolution media file may be partitioned into multiple subfiles based on the keyframes, and the partitioned media data and corresponding metadata may be packaged into corresponding target resolution segmented media files, where each segmented media file can be independently decoded.

Here, preloading means that the player adds the target resolution media file segment to a Buffer (Source Buffer) of a media Source of the MSE, and the media element of the web page is played according to a virtual address (the media Source is used as a data Source). The playing request can be an HTTP-based request initiated by the player through a webpage, and can also be an HTTPS-based request.

step 1503: when the segmented media file with the target resolution is obtained and the next segmented media file with the original resolution is played, the player continues to request to obtain the next segmented media file with the target resolution for continuing the real-time playing point until the next segmented media file with the target resolution is retrieved before the playing of the next segmented media file with the original resolution is finished.

Step 1504: and loading the next segment media file with the target resolution through the player so as to play the next segment media file with the original resolution in the webpage in a continuous manner.

Here, the player loading means loading into a cache of the player. For the pre-loaded segmented media files of the original resolution and the segmented media files of the target resolution, the segmented media files can be retrieved through the playing requests, and one segmented media file can be retrieved through each playing request; that is, for the whole media file, the data retrieved each time can be encapsulated into a virtual URL object through one or more playback requests, so that the media elements of the web page can deliver the virtual URL to play the segmented media file.

Fig. 16 is a schematic view of another alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure; the continuous playing method during resolution switching can be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 16, a flow of implementing another continuous playing method during resolution switching in the embodiment of the present disclosure will be described with reference to the steps shown in fig. 16.

Step 1601: when the player plays the segmented media file in the embedded webpage, the player detects a real-time playing point for playing the segmented media file with the original resolution when receiving a resolution switching event.

step 1602: in the process of playing the segmented media files with the original resolution on the webpage, at least one segmented media file with the original resolution behind the real-time playing point is preloaded through the player.

here, the play request may be an HTTP-based request initiated by the player through a web page, or may be an HTTPs-based request.

It is to be noted that the metadata corresponding to the media data refers to the metadata of the segment level corresponding to the media data. The media data in the target resolution media file may be partitioned into a plurality of subfiles based on the keyframes, and the partitioned media data and corresponding metadata may be packaged into corresponding target resolution segmented media files, where each segmented media file is capable of being independently decoded.

Step 1603: and when the real-time playing point of the segmented media file with the original resolution is the end time of the currently played segmented media file with the original resolution, continuously playing the next segmented media file with the preloaded original resolution in the webpage by using the player.

Fig. 17 is a schematic view of another alternative implementation flow of a continuous playing method during resolution switching according to an embodiment of the present disclosure; the continuous playing method during resolution switching can be applied to various types of players, such as an H5 player embedded in a web page loaded by a browser or an application program playing a media file; as shown in fig. 17, regarding another implementation flow of the continuous playing method during resolution switching in the embodiment of the present disclosure, the following steps will be described with reference to fig. 17.

step 1701: when the player receives the playing requests from the web pages, the player stores the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests.

Step 1702: and the player allocates connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence.

here, the connection is for the player to request a segmented media file to be played.

In the embodiment of the present disclosure, the number of connections allocated for the play request in the buffer queue does not exceed the upper limit of the number of concurrent connections. The number of concurrent connections that can be used by the buffer queue may be used to indicate the number of connections that can be simultaneously used by the buffer queue, and generally, when the upper limit of the concurrent connection data of the buffer queue is not reached, each play request can be allocated with one connection to request media data from the server. For example, when the delay tolerance of the user to other services (such as non-video playing or web browsing) in the web page is higher, the value of the configurable concurrent connection number is larger than that of the configurable concurrent connection number when the delay tolerance of some services is lower.

For the player in this step 1702 to allocate connections for the playing requests in the buffer queue according to the upper limit of the number of concurrent connections that can be used by the buffer queue and the receiving order, at least one of the following two ways may be adopted:

Step 1703: when the player plays the segmented media file through the connection request in the embedded webpage, the real-time playing point of the segmented media file with the original resolution is detected when the resolution switching event is received.

Step 1704: and the player acquires the segmented media file for continuing the real-time playing point from the server, wherein the segmented media file for continuing the real-time playing point is formed by the media file based on the target resolution.

In the embodiment of the present disclosure, for the player in this step 1704 to obtain the segmented media file for continuing the live play point from the server, the following manner may be adopted to implement: first, the player requests metadata in a media file of a target resolution from the server; then, the player determines two key frames for continuing the real-time playing point according to the media information identified from the metadata, and requests the media data between the two key frames from the server; finally, the player encapsulates the requested media data and corresponding metadata to form a segmented media file of a corresponding target resolution.

for example, when the target resolution media file is an MP4 file (file that does not support streaming media playback), the target resolution segmented media file is formed by extracting video frames between two key frames and aligned audio frames from the target resolution media file based on the two key frames determined in the target resolution media file, calculating corresponding metadata, and filling in according to the packaging format of FMP 4.

Step 1705: when the segmented media file of the target resolution ratio of the continuous real-time playing point is obtained, the player clears the playing request aiming at the original resolution ratio in the buffer queue and cancels the connection already distributed aiming at the playing request aiming at the original resolution ratio.

In another optional embodiment of the present disclosure, when the segmented media file of the target resolution succeeding the live playing point is acquired, the player empties the pre-cached segmented media file of the original resolution succeeding the live playing point.

In summary, the embodiments of the present disclosure have the following beneficial effects: the method overcomes the network delay caused by the fact that the media file with the target resolution needs to be requested again, enables the whole media file to be played smoothly, further avoids the phenomenon of playing card pause, can effectively improve the efficiency of retrieving the media data fragment with the target resolution from the webpage, and greatly improves the watching experience of a user.

The technical solutions described in the embodiments of the present disclosure can be arbitrarily combined without conflict.

the above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present disclosure, and all the changes or substitutions should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method for continuous playback during resolution switching, the method comprising:

2. The method of claim 1, further comprising:

in the process of playing the segmented media file with the original resolution, at least one segmented media file with the original resolution behind the real-time playing point is pre-loaded through the player;

When the segmented media file with the target resolution is acquired and the next segmented media file with the original resolution is already played,

And continuing to request to acquire the next segmented media file with the target resolution ratio for continuing the real-time playing point until the next segmented media file with the original resolution ratio is played to the end, and retrieving the next segmented media file with the target resolution ratio.

3. The method of claim 1, further comprising:

When the real-time playing point of the segmented media file with the original resolution is the end time of the currently played segmented media file with the original resolution,

And continuing to play the next segmented media file with the preloaded original resolution on the webpage through the player.

4. The method of claim 1, further comprising:

When a playing request is received through the player, storing the received playing request in a buffer queue for queuing according to the receiving sequence of the playing request;

Distributing connection for the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence;

wherein the connection is for the player to request a segmented media file to be played.

5. The method of claim 4, further comprising:

When the segmented media file of the target resolution succeeding a real-time playing point is acquired,

Emptying the play request for the original resolution in the buffer queue, and,

Canceling the connection already allocated for the play request of the original resolution.

6. The method of claim 4, further comprising:

and emptying the pre-cached segmented media file with the original resolution ratio for continuing the real-time playing point.

7. The method of claim 1, wherein obtaining the segmented media file for succeeding the live playback point comprises:

requesting metadata in a media file at a target resolution;

Determining two key frames for continuing the real-time playing point according to the media information identified from the metadata so as to request the media data between the two key frames;

and packaging and forming a segmented media file with a corresponding target resolution according to the requested media data and the corresponding metadata.

8. a playback apparatus for switching resolution, the apparatus comprising: the device comprises a detection module, an acquisition module and a loading module; wherein the content of the first and second substances,

9. The apparatus of claim 8,

The loading module is further configured to pre-load, by the player, at least one segmented media file with an original resolution after the real-time playing point in a process of playing the segmented media file with the original resolution;

The obtaining module is further configured to, when the segmented media file with the target resolution is obtained and the next segmented media file with the original resolution is already being played, continue to request to obtain the next segmented media file with the target resolution for continuing the live play point until the next segmented media file with the target resolution is played to the end, retrieve the next segmented media file with the target resolution.

10. the apparatus of claim 8,

The device further comprises: and the playing module is used for continuing to play the next segmented media file with the original resolution which is pre-loaded on the webpage through the player when the real-time playing point of the segmented media file with the original resolution is the end time of the segmented media file with the original resolution which is currently played.

11. the apparatus of claim 8, further comprising:

The storage module is used for storing the received playing requests in a buffer queue for queuing according to the receiving sequence of the playing requests when the playing requests are received by the player;

The distribution module is used for distributing connection to the playing requests in the buffer queue according to the upper limit of the number of the concurrent connections which can be used by the buffer queue and the receiving sequence;

12. The apparatus of claim 11, further comprising:

A management module for, when a segmented media file of the target resolution is acquired that continues a live play point,

Emptying the play request for the original resolution in the buffer queue, and,

13. the apparatus of claim 11,

The management module is further configured to, when the segmented media file of the target resolution is acquired to continue the live play point,

14. The apparatus of claim 8, wherein the obtaining module is specifically configured to:

Requesting metadata in a media file at a target resolution;

15. A continuous playing device during resolution switching comprises:

a memory for storing executable instructions;

A processor, configured to execute the executable instructions to implement the method for continuous playback during resolution switching according to any one of claims 1 to 7.

16. A storage medium storing executable instructions for implementing a sequential playback method at resolution switching according to any one of claims 1 to 7 when executed.