WO2020078165A1 - 视频处理方法、装置、电子设备及计算机可读介质 - Google Patents

视频处理方法、装置、电子设备及计算机可读介质 Download PDF

Info

Publication number
WO2020078165A1
WO2020078165A1 PCT/CN2019/107074 CN2019107074W WO2020078165A1 WO 2020078165 A1 WO2020078165 A1 WO 2020078165A1 CN 2019107074 W CN2019107074 W CN 2019107074W WO 2020078165 A1 WO2020078165 A1 WO 2020078165A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
audio
stream
module
decoder
Prior art date
Application number
PCT/CN2019/107074
Other languages
English (en)
French (fr)
Inventor
胡小朋
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to EP19873400.6A priority Critical patent/EP3866477A4/en
Publication of WO2020078165A1 publication Critical patent/WO2020078165A1/zh
Priority to US17/231,774 priority patent/US11336953B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H04N21/42607—Internal components of the client ; Characteristics thereof for processing the incoming bitstream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/418—External card to be used in combination with the client device, e.g. for conditional access
    • H04N21/4183—External card to be used in combination with the client device, e.g. for conditional access providing its own processing capabilities, e.g. external module for video decoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/426—Internal components of the client ; Characteristics thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4341—Demultiplexing of audio and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4396—Processing of audio elementary streams by muting the audio signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/8166—Monomedia components thereof involving executable data, e.g. software
    • H04N21/8193—Monomedia components thereof involving executable data, e.g. software dedicated tools, e.g. video decoder software or IPMP tool
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format

Definitions

  • the present application relates to the technical field of video processing, and more specifically, to a video processing method, device, electronic device, and computer-readable medium.
  • This application proposes a video processing method, device, electronic device, and computer-readable medium to improve the above-mentioned defects.
  • an embodiment of the present application provides a video processing method, which is applied to an electronic device, and the electronic device includes a system playback module and an FFmpeg module.
  • the method includes: the system playback module acquiring the video file to be played sent by the target client; judging whether the video file to be played is supported; if not, parsing the video file to obtain the video to be played Audio stream and video stream in the file; configure the audio decoder ID for the audio stream and configure the video decoder ID for the video stream; decode the video stream, the audio stream, the video decoder ID and the audio
  • the identifier of the device is sent to the FFmpeg module, so that the FFmpeg module finds the corresponding video decoder according to the identifier of the video decoder and decodes the video stream to obtain video playback data, and decodes according to the audio
  • the identifier of the device finds the corresponding audio decoder and decodes the audio stream to obtain audio playback data, wherein the audio play
  • an embodiment of the present application also provides a video processing method, which is applied to an electronic device, and the electronic device includes a system playback module and an FFmpeg module.
  • the method includes: the system playback module obtains the video file to be played sent by the target client; the system playback module determines whether the video file to be played is supported; if it does not support, the system playback module will Parse the video file to obtain the audio stream and the video stream in the video file to be played, configure the audio decoder ID for the audio stream and configure the video decoder ID for the video stream; the system playback module converts the video Stream, the audio stream, the identifier of the video decoder and the identifier of the audio decoder are sent to the FFmpeg module; the FFmpeg module decodes the video file to be played to obtain audio playback data and video playback data, wherein The audio playback data and video playback data can be played by the audio and video output module of the electronic device.
  • an embodiment of the present application further provides a video processing apparatus, including: an acquisition unit, a judgment unit, an analysis unit, a configuration unit, and a processing unit.
  • the obtaining unit is used to obtain the video file to be played sent by the target client.
  • the judging unit is used to judge whether to support playing the video file to be played.
  • the parsing unit is used to parse the video file to be played to obtain the audio stream and the video stream in the video file to be played if it is not supported.
  • the configuration unit is configured to configure the identifier of the audio decoder for the audio stream and the identifier of the video decoder for the video stream.
  • the processing unit is configured to send the identification of the video stream, the audio stream, the video decoder, and the identification of the audio decoder to the FFmpeg module, so that the FFmpeg module finds according to the identification of the video decoder A corresponding video decoder and decodes the video stream to obtain video playback data, and finds a corresponding audio decoder according to the identifier of the audio decoder and decodes the audio stream to obtain audio playback data, wherein, The audio playback data and video playback data can be played by the audio and video output module of the electronic device.
  • an embodiment of the present application further provides a video processing device, including a system playback module and an FFmpeg module.
  • the system play module is used to obtain the video file to be played sent by the target client; obtain the target type of the video file to be played; determine whether to support playing the video file of the target type; Parse the video file to obtain the audio stream and the video stream in the video file to be played, configure the audio decoder ID for the audio stream and configure the video decoder ID for the video stream, and convert the video stream and the audio
  • the identifiers of the stream, the video decoder, and the audio decoder are sent to the FFmpeg module.
  • the FFmpeg module is used to find a corresponding video decoder according to the identifier of the video decoder and decode the video stream to obtain video playback data, and find a corresponding audio decoder according to the identifier of the audio decoder And decode the audio stream to obtain audio playback data, wherein the audio playback data and video playback data can be played by the audio and video output module of the electronic device.
  • an embodiment of the present application further provides an electronic device, including: an image processor, a memory, a system playback module, an FFmpeg module, and one or more application programs.
  • an electronic device including: an image processor, a memory, a system playback module, an FFmpeg module, and one or more application programs.
  • the one or more application programs are stored in the memory and configured to be executed by the image processor, and the one or more program programs are configured to perform the above method.
  • an embodiment of the present application further provides a computer-readable storage medium.
  • the computer-readable storage medium stores program code, and the program code may be called by a processor to execute the above method.
  • the solution provided by the present application calls the system playback module to play the video file when the client plays the video file to be played, and then the system playback module obtains the video file to be played, according to the video file to be played
  • the type determines whether to support the playback of this type of video file. If it does not, the video file to be played is parsed into a video stream and an audio stream, and the corresponding decoder IDs are respectively configured and sent to the FFmpeg module.
  • the FFmpeg module decodes the audio stream and the video stream according to their respective decoders to obtain audio playback data and video playback data, wherein the audio playback data and video playback data can be played by the audio and video output module of the electronic device,
  • the audio and video system of the electronic device can play audio playback data and video playback data. Therefore, by calling the FFmpeg module, video files that do not support playback can be played smoothly.
  • FIG. 1 shows a block diagram of a video playback architecture provided by an embodiment of the present application
  • FIG. 2 shows a method flowchart of a video processing method provided by an embodiment of the present application
  • FIG. 3 shows a flowchart of the method of S204 in FIG. 1;
  • FIG. 4 shows a method flowchart of a video processing method provided by another embodiment of the present application.
  • FIG. 5 shows a schematic diagram of an interface change when a client plays a video provided by an embodiment of the present application
  • FIG. 6 shows a schematic diagram of an interface change when a client plays a video provided by another embodiment of the present application.
  • FIG. 7 shows a method flowchart of a video processing method provided by another embodiment of the present application.
  • FIG. 8 shows a method flowchart of a video processing method provided by yet another embodiment of the present application.
  • FIG. 9 shows a block diagram of a video processing device provided by an embodiment of the present application.
  • FIG. 10 shows a block diagram of a video processing apparatus provided by another embodiment of the present application.
  • FIG. 11 shows a structural block diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 12 shows a storage unit for storing or carrying program codes for implementing a video processing method according to an embodiment of the present application.
  • FIG. 1 shows a block diagram of a video playback architecture.
  • the system includes a client 10, a system playback module 20 and an audio and video output module 30.
  • the client 10 sends the video file to the system playback module 20.
  • the system playback module 20 obtains the data to be played, the next step is to analyze the audio and video data.
  • General video files are composed of two parts: video stream and audio stream.
  • the audio and video packaging formats of different video formats are definitely different.
  • the process of synthesizing audio streams and video streams into files is called muxer, while the process of separating audio streams and video streams from media files is called demuxer.
  • the system playback module 20 is a default playback module in the operating system of the electronic device, that is, when a client installed in the electronic device plays a video file, the system playback module 20 is called to play the video file by default.
  • the system playback module 20 may be a Media Framework module.
  • the parsing submodule 201 in the system playback module 20, where the parsing submodule 201 may be an Android system.
  • the parsing submodule 201 may be an Android system.
  • the MediaExtractor module in addition to the process of parsing the video file into a video stream and an audio stream, you also need to configure the corresponding decoder for the video stream and audio stream, that is, configure the corresponding decoding strategy.
  • the decoded video frame can be directly rendered, and the audio frame can be sent to the buffer of the audio output device for playback. Of course, the timestamp of video rendering and audio playback must be synchronized.
  • the decoding operation is performed by the system playback module 20.
  • the decoding sub-module 202 executes.
  • the decoding sub-module 202 may be MediaCodec in the Android system.
  • video decoding may include hard decoding and soft decoding.
  • hardware decoding a part of the video data that is originally handed over to the central processing unit (Central Processing Unit, CPU) for processing is handed over to the graphics processor (Graphics Processing Unit, GPU) To do it, and the GPU's parallel computing power is much higher than that of the CPU, which can greatly reduce the load on the CPU. After the CPU occupancy rate is low, you can run some other programs at the same time.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the hardware solution and software are selected according to the requirements.
  • the decoded audio stream becomes audio playback data
  • it may be PCM format audio data
  • the audio and video output module 30 drives the voice coil and other components of the audio playback device to vibrate and sound according to the PCM encoding.
  • the video stream after decoding becomes video playback data.
  • it can be YUV format video data.
  • the audio and video output module 30 sends the YUV format video data to the layer transfer module (SurfaceFlinger), and the SurfaceFlinger will decode the video data. After the video data is rendered and synthesized, it is displayed on the display.
  • SurfaceFlinger is an independent Service, which receives all Window's Surface as input, calculates the position of each Surface in the final composite image according to the parameters such as ZOrder, transparency, size, position, etc., and then passes it to HWComposer or OpenGL to generate the final The display buffer is then displayed on a specific display device.
  • the system playback module does not support playback of video files in some formats, for example, MP4 format. If playback of video files in this format is not supported, Call a third-party application to play, that is to say, when using the system playback module, a video file cannot be decoded, you can call a third-party application to soft decode the video file, so as to obtain audio that can be played by the audio and video output module And video data.
  • some formats for example, MP4 format.
  • an embodiment of the present application provides a video processing method, which is applied to the above electronic device, and the electronic device includes a system playback module and an FFmpeg module.
  • the main body of the method If it is a system playback module, the method includes: S201 to S204.
  • the system playback module obtains the video file to be played sent by the target client.
  • the client is an application program installed in the electronic device, and has a video playing function.
  • the client has an icon on the system desktop.
  • the user can open the client by clicking the icon of the client.
  • the package name of the application that the user clicks can be confirmed.
  • the package name of the video application can be obtained from the code in the background of the system.
  • the name format is: com.android.video.
  • the video list interface of the client displays the display content corresponding to multiple videos.
  • the display content corresponding to multiple videos includes a thumbnail corresponding to each video.
  • the thumbnail can be used as a touch button. The user clicks the thumbnail. It can detect the thumbnail that the user wants to click, and can also determine the video file to be played.
  • the client responds to the video selected by the user in the video list, enters the video playback interface, and clicks the play button on the playback interface.
  • the client can detect what type of video the user is currently clicking by monitoring the user's touch operation Specifically, the play button is set with a preset attribute, and it is possible to determine the video to be played selected by the user by detecting the acquired attribute of the play button corresponding to the touch operation.
  • the client After acquiring the to-be-played video file input by the user, the client sends the to-be-played video file to the system play module, so that the system play module parses and decodes the video file and plays it.
  • S202 Determine whether to support playing the video file to be played.
  • the system playback module can parse the supported video files into video streams and audio streams, and can successfully decode the video streams and audio streams to obtain audio playback data and video playback data, wherein the audio playback data and video playback data It can be played by the audio and video output module of the electronic device.
  • the audio playback data and the video playback data are audio PCM data and video YUV data, respectively.
  • the system playback module cannot parse the video file, that is, the corresponding video stream and audio stream cannot be obtained from the video file, or even if the video file can be parsed normally, That is, the corresponding video stream and audio stream can be obtained from the video file, and because the video stream and audio stream in the format cannot be decoded, the audio playback data and video playback data cannot be obtained, and the video file cannot be played normally.
  • the target type of the video file to be played is obtained.
  • the type of the video file can be determined by the extension of the video file. For example, if the extension is .mp4, the target type of the video file to be played is mp4.
  • the type of the video file may be the format of the video file.
  • the system playback module corresponds to a format list of video files that support playback.
  • the format list includes multiple format identifiers, and the video files corresponding to each format in the format list can be played normally.
  • the The format list can include the format of the audio stream and the format of the video stream corresponding to the format of the video file, as well as the format of each video stream and the identifier of the decoder corresponding to the format of the audio stream. Configure the correct decoder for the supported video formats.
  • the format of the video file is the type of the video file
  • the format of the video stream is the type of the video stream
  • the format of the audio stream is the type of the audio stream.
  • the target type of the video file After obtaining the target type of the video file, by searching the format list, it is determined whether the target type of the video file matches the type of the video file in the format list, and if there is a matching type, it is determined to support playback If there is no matching type of the target type video file, it is determined that the target type video file is not supported.
  • S203 Parse the video file to be played to obtain the audio stream and the video stream in the video file to be played.
  • the system playback module uses the parsing submodule to parse the video file to be played to obtain the corresponding video stream and audio stream. For example, it can be obtained through the above-mentioned demuxer process, which can decapsulate the video file to be played and obtain the video to be played. The video stream and audio stream corresponding to the video file.
  • S204 Configure an identifier of the audio decoder for the audio stream and configure an identifier of the video decoder for the video stream.
  • the steps of S204 may include: S2041 to S2043.
  • S2041 Determine the type of the video stream and the type of the audio stream.
  • the type of the video stream and the type of the audio stream can also be determined by acquiring the extensions of the video stream and the audio stream.
  • S2042 Configure an identifier of the video decoder for the video stream according to the type of the video stream.
  • a first correspondence between the type of video stream and the identifier of the video decoder is obtained in advance, and the first correspondence can be manually input by the user, for example, according to the introduction of a video decoder that has determined that it can decode a video file of this type Enter the corresponding relationship, for example, the first corresponding relationship is obtained from a network server, and the network server counts the identifiers of decoders capable of decoding each type of video stream.
  • the first correspondence can also be obtained by the FFmpeg module. Specifically, it can be the decoder corresponding to each type of video stream in the format list corresponding to the FFmpeg module when the FFmpeg module is installed.
  • the first correspondence relationship is set by marking.
  • the identifier of the decoder capable of decoding the video stream of the type is searched for by the first correspondence, so as to configure the identifier of the video decoder for the video stream.
  • S2043 Configure an identifier of the audio decoder for the audio stream according to the type of the audio stream.
  • the second correspondence between the type of audio stream and the identifier of the audio decoder is also obtained in advance, and the second correspondence can also be obtained in the manner of acquiring the first correspondence.
  • the second correspondence relationship is used to search for the identifier of the audio decoder capable of decoding the audio stream of this type, so as to configure the identifier of the audio decoder for the audio stream.
  • S205 Send the identifier of the video stream, the audio stream, the video decoder, and the identifier of the audio decoder to the FFmpeg module, so that the FFmpeg module finds the corresponding video according to the identifier of the video decoder
  • the decoder decodes the video stream to obtain video playback data, and finds a corresponding audio decoder according to the identifier of the audio decoder and decodes the audio stream to obtain audio playback data.
  • the audio playback data and video playback data can be played by the audio and video output module of the electronic device.
  • the FFmpeg module is an open source cross-platform video and audio streaming framework, which is free software and uses the LGPL or GPL license (depending on the selected component). It provides a complete solution for recording, converting, and streaming audio and video. It contains a rich audio / video codec library libavcodec.
  • the FFmpeg module is composed of the following parts:
  • 2.ffserver is a multimedia server for real-time broadcasting based on HTTP and RTSP.
  • 3.ffplay is a simple media player developed with SDL and FFmpeg libraries.
  • libavcodec is a library that contains all FFmpeg audio and video codecs.
  • libavformat is a library that contains all parsers and generators for common audiovisual formats.
  • the FFmpeg module integrates multiple formats of audio and video encoders, and has audio and video playback functions, as well as video file analysis functions.
  • the FFmpeg module is injected into the operating system of the electronic device, and the calling interface between the system playback module and the FFmpeg module is defined, for example, it can be an API or an SDK, so that the system playback module and the FFmpeg module can communicate .
  • the system playback module determines that it does not support the normal playback of the video file requested by the client, the video file is sent to the FFmpeg module, and due to the powerful encoding and decoding capabilities of the FFmpeg module, it can support the analysis of a large number of format video files and decoding.
  • the FFmpeg module can be embedded in the operating system, and the FFmpeg module can be used as a service process to decode video files in the background of the system, thereby generating audio playback data and video playback data without having to switch to the foreground, which allows customers The client is forced to switch to the background and avoid interrupting the user's operation of the client.
  • the audio playback data and video playback data are obtained in the FFmpeg module, and the audio playback data and video playback data can be sent to the system playback module, and the system playback module controls the audio and video output module of the electronic device, using the audio playback device (for example, Speakers, earpieces, headphones, etc.) play audio playback data, and use video playback devices (eg, screens) to play video playback data.
  • the audio playback device for example, Speakers, earpieces, headphones, etc.
  • video playback devices eg, screens
  • the FFmpeg module can also directly control the audio and video output module of the electronic device to output audio playback data and video playback data.
  • the system playback module can directly deliver the video file to the FFmpeg module, and the FFmpeg module can parse and decode the video file to be played. After parsing the video file to be played, the video stream and the audio stream are obtained and then sent to the FFmpeg module for decoding.
  • the system playback module After acquiring the identifier of the video decoder corresponding to the video stream and the identifier of the audio decoder corresponding to the audio stream, the system playback module packages and sends it to the FFmpeg module, specifically, the video stream and the audio
  • the stream is stored in the storage space of the electronic device, and address identifiers such as handles or pointers are defined.
  • the process of sending the video stream and the audio stream to the FFmpeg module is to send the address identifier of the audio stream and the address identifier of the video stream to FFmpeg module
  • the FFmpeg module can obtain the audio stream corresponding to the address identifier from the storage space of the electronic device according to the address identifier of the audio stream. Similarly, it can also be obtained from the storage space of the electronic device according to the address identifier of the video stream This address identifies the corresponding video stream.
  • the FFmpeg module determines the video decoder corresponding to the identifier of the video decoder and the audio decoder corresponding to the identifier of the audio decoder according to the correspondence relationship between the identifiers of the decoders and the decoders stored in advance. Then, the video decoder is called to decode the video stream to obtain video playback data, and the audio decoder is called to decode the audio stream to obtain audio playback data. Therefore, by calling the FFmpeg module, those video files that do not support playback can be played smoothly.
  • the video processing method includes: S401 to S413.
  • the video file to be played is sent to the system playback module through the API interface with MediaEtractor, that is, under the default playback process of the client, the client sends the video data to be played to MediaEtractor
  • MediaEtractor For analysis, there is an API interface between MediaEtractor and the client.
  • S402 Send the video file to be played to MPEG4 Extractor.
  • the MediaEtractor obtains the video file to be played, it is determined that the video file to be played is of the MP4 type, and then the MPEG4Extractor is selected to parse the MP4 type video file, and then the video file to be played is sent to the MPEG4Extractor, where MediaEtractor and MPEG4Extractor All are program modules embedded in the system playback module.
  • S403 Determine whether to support playing MP4 format video files.
  • the system playback module judges whether to support playing MP4 format video files, specifically, it may be MPEG4Extractor to judge whether to support playing MP4 format video files.
  • MediaExtractor can be used to parse the video data to obtain the video stream and audio stream, and then use MediaCodec to decode the video stream and audio stream to obtain audio PCM data and video YUV data processing, and then send them to the audio and video output systems for output. .
  • S405 Parse the video file to be played through MPEG4Extractor.
  • the MPEG4 Extractor is used to parse the video file to be played.
  • the type of the video file is determined to be the MP4 format, and a type ID is configured for the video file to be played, for example, the type ID is mimetype.
  • the type ID is used to indicate the type of the video file.
  • the type ID defined for the MP4 format is inconsistent with the type ID recorded in the system playback module, thereby preventing the system playback module from using the type
  • the video file corresponding to the ID is parsed using native parsing logic and an error occurs, because in the parsing process, in addition to decapsulating the video file into a video stream and an audio stream, a video decoder for the video stream and audio for the audio stream Decoder, if using native parsing logic to parse, even if the video stream and audio stream will be obtained, the corresponding decoder cannot be successfully configured, and an error occurs, and the system playback module cannot recognize the type ID defined in this application, namely mimetype, will not parse and decode the type ID. Then, for the type ID of mimetype, the parsing and decoding logic corresponding to the embodiments of the present application can be used, instead of the native parsing logic.
  • the audio stream is configured with an audio / ffmpeg identity identifier, which is used as the audio stream identity identifier, which is convenient for determining the audio stream identity identifier when configuring the decoder
  • the analysis and decoding method of this application is selected, that is, the video processing method using the FFmpeg module, that is, the identity of the audio stream corresponds to the decoding strategy.
  • the audio stream format includes mp3 and adpcm, it is determined whether the audio stream format is mp3 or adpcm, if it is mp3, then execute S408, if it is adpcm, then execute S409.
  • the identifier of the audio decoder configuring the audio stream is FFMPEG_CODEC_ID_MP3.
  • program code for parsing MP4 format can be added in MPEG4 Extractor, for example, code for parsing such chunk type is added, so that MPEG4Extractor can parse and configure the decoder for the MP4 format video file through the added code.
  • the identifier of the configured audio decoder is FFMPEG_CODEC_ID_MP3
  • the decoder corresponding to the identifier can decode the audio stream in MP3 format.
  • the decoder ID is parsed and configured as follows:
  • Audio stream mp3 Chunk type is .mp3, configure audio stream MimeType to audio / ffmpeg, CodecID to FFMPEG_CODEC_ID_MP3.
  • FFMPEG_CODEC_ID_MP3 is the identifier of the audio decoder, and assigns a value to the identifier of the audio decoder, that is, 0x15001, when the identifier of the audio decoder is sent to FFmpeg, FFMPEG_CODEC_ID_MP3 and the assigned value 0x15001 are sent to FFmpeg together, then FFmpeg can find the corresponding audio decoder as an MP3 decoder through FFMPEG_CODEC_ID_MP3, and can also find the corresponding decoder according to 0x15001.
  • AV_CODEC_ID_ADPCM_MS 0x11006;
  • AV_CODEC_ID_MJPEG 0x8;
  • AV_CODEC_ID_MP3 0x15001
  • AV_CODEC_ID_SVQ3 0x18.
  • the assignment of the audio decoder configured for the MP3 format audio stream is the same as the assignment of the MP3 format audio decoder in FFmpeg, for example, both are 0X15001, then 0X15001 can quickly find the MP3 audio decoder.
  • FFMPEG_CODEC_ID_MP3 it can also be determined that the audio decoder corresponding to the identifier is also an MP3 audio decoder. For example, by extracting the keyword of the identifier, the text "MP3" can be extracted. Determine that the logo corresponds to the MP3 audio decoder.
  • the identifier of the audio decoder configured with the audio stream is FFMPEG_CODEC_ID_ADPCM_MS.
  • the identifier of the configured audio decoder of the audio stream is FFMPEG_CODEC_ID_ADPCM_MS, and the decoder corresponding to the identifier can decode the audio stream of adpcm format.
  • the decoder ID is parsed and configured as follows:
  • Audio stream adpcm Chumk type is ms02, configure audio stream MimeType to audio / ffmpeg, CodecID to FFMPEG_CODEC_ID_ADPCM_MS.
  • FFMPEG_CODEC_ID_ADPCM_MS is the identifier of the audio decoder, and assigns a value to the identifier of the audio decoder, that is, a value of 0x11006, when sending the identifier of the audio decoder to FFmpeg, FFMPEG_CODEC_ID_ADPCM_MS and the assigned value 0x11006 are sent to FFmpeg FFmpeg can find the corresponding audio decoder as the adpcm decoder through FFMPEG_CODEC_ID_ADPCM_MS, and can also find the corresponding decoder according to 0x11006.
  • the specific process is the same as the above MP3 process, which will not be repeated here.
  • the video stream is configured with an identity tag video / ffmpeg, which is used as the identity of the video stream, which is convenient for determining the identity of the video stream when configuring the decoder
  • the analysis and decoding method of this application is selected, that is, the video processing method using the FFmpeg module, that is, the identity of the video stream corresponds to the decoding strategy.
  • the video stream format includes mjpeg and sorenson, it is determined whether the audio stream format is mjpeg or sorenson, if it is mjpeg, then execute S412, if it is sorenson, then execute S413.
  • S412 The identifier of the video decoder configuring the video stream is FFMPEG_CODEC_ID_MJPEG.
  • the identifier of the configured video decoder of the video stream is FFMPEG_CODEC_ID_MJPEG, and the decoder corresponding to the identifier can decode the video stream in the mjpeg format.
  • the decoder ID is parsed and configured as follows:
  • Video stream mjpeg Chunk type is jpeg, configure video stream MimeType to video / ffmpeg, CodecID to FFMPEG_CODEC_ID_MJPEG.
  • FFMPEG_CODEC_ID_MJPEG is the identifier of the video decoder, and assigns a value to the identifier of the video decoder, that is, 0x8.
  • FFMPEG_CODEC_ID_MJPEG is the identifier of the video decoder, and assigns a value to the identifier of the video decoder, that is, 0x8.
  • FFmpeg can find the corresponding video decoder as mjpeg decoder through FFMPEG_CODEC_ID_MJPEG, and can also find the corresponding decoder according to 0x8.
  • the specific process is the same as the above MP3 process, and will not be repeated here.
  • S413 The identifier of the video decoder configuring the video stream is FFMPEG_CODEC_ID_SORENSON.
  • the identifier of the configured video decoder of the video stream is FFMPEG_CODEC_ID_SORENSON, and the decoder corresponding to the identifier can decode the video stream in the sorenson format.
  • the decoder ID is parsed and configured as follows:
  • Video stream sorenson Chunk type is sorenson, configure video stream MimeType to video / ffmpeg, CodecID to FFMPEG_CODEC_ID_SORENSON.
  • FFMPEG_CODEC_ID_SORENSON is the identifier of the video decoder, and assigns a value to the identifier of the video decoder, that is, 0x18.
  • FFMPEG_CODEC_ID_SORENSON and the assigned value 0x18 are sent to FFmpeg together FFmpeg can find the corresponding video decoder as the sorenson decoder through FFMPEG_CODEC_ID_SORENSON, and can also find the corresponding decoder according to 0x18.
  • the specific process is the same as the above MP3 process, which will not be repeated here.
  • the unsupported videos of this type can be parsed normally, and binary data to be decoded can be obtained.
  • APP calls MediaCodec and sends out the correct audio and video stream MimeType, selects the integrated ffmpeg decoder system in the system, and accurately configures the relevant decoder in ffmpeg through CodecID.
  • the client when playing video files that are not supported by the system playback module, the client can avoid switching to the video playback interface of the third-party application by calling the third-party application to softly decode the video file to play the video. Instead, use the system playback module to play video files without jumping.
  • the screen of FIG. 5 (a) is the interface of the client, and a play button corresponding to a video file to be played, that is, a play button corresponding to "video content" is displayed in the interface, and the user clicks the play button.
  • the third-party application is called to play the video
  • the client will be switched to the background, and the third-party application is called and switched to the foreground, as shown in FIG. 5 (b).
  • the system playback module and FFmpeg will parse and decode the video in the background, which will not cause the client to be switched to the background, as shown in FIG. 6 (b), still on the interface of the client Play the video.
  • an embodiment of the present application provides a video processing method, which is applied to the foregoing electronic device, and the electronic device includes a system playback module and an FFmpeg module.
  • the execution subject of the method is system playback Module, the method includes: S701 to S708.
  • the system playback module obtains the video file to be played sent by the target client.
  • S702 Determine whether to support playing the video file to be played.
  • S703 Parse the video file to be played to obtain the audio stream and the video stream in the video file to be played.
  • S704 Configure the identifier of the audio decoder for the audio stream and configure the identifier of the video decoder for the video stream.
  • S705 Send the identifier of the video stream, the audio stream, the video decoder, and the identifier of the audio decoder to the FFmpeg module, so that the FFmpeg module finds the corresponding video according to the identifier of the video decoder
  • the decoder decodes the video stream to obtain video playback data, and finds a corresponding audio decoder according to the identifier of the audio decoder and decodes the audio stream to obtain audio playback data.
  • S706 Acquire the audio playback data and video playback data returned by the FFmpeg module.
  • the audio playback data and video playback data are obtained in the FFmpeg module, and the audio playback data and video playback data can be sent to the system playback module, and the system playback module controls the audio and video output module of the electronic device, using audio playback devices (for example, speakers, Earpieces or headphones, etc.) to play audio playback data, and use video playback devices (eg, screens) to play video playback data.
  • audio playback devices for example, speakers, Earpieces or headphones, etc.
  • video playback devices eg, screens
  • the FFmpeg module can also directly control the audio and video output module of the electronic device to output audio playback data and video playback data.
  • the FFmpeg module returns the decoded audio playback data and video playback data to the system playback module, then the system playback module can control the audio and video output module of the electronic device to play the audio playback data and video playback Data, therefore, the system playback module continues to use the native playback logic to play the video, which is more portable.
  • S707 Detect whether the target client is in a silent mode.
  • the target client is a client that requests to play a video file to be played.
  • a method for confirming whether the target client is in the silent mode is to obtain a volume parameter of the target client, and if the volume parameter is the target parameter, determine that the target client is in the silent mode. If the volume parameter is not the target parameter, it is determined that the target client is not in the silent mode.
  • the first parameter corresponds to the mute state
  • the second parameter corresponds to the non-mute state
  • the target parameter is the first parameter.
  • the target client reads the volume parameter corresponding to the target client. If the read volume parameter is the first parameter, that is, the volume parameter is the target parameter, the target client is muted. When the application's volume state is mute. If the volume parameter is the second parameter, that is, the volume parameter is not the first parameter, the target client is instructed to play the video file according to a preset volume, that is, the volume state of the application is now in a non-silent state.
  • the specific implementation method of muting the target client may be that the video file in the target client is prohibited from playing, or the video file may be played in silent mode, that is, the video file is run but does not pass through a speaker or a headset, etc. Play the video file.
  • the volume parameter of the target client may be set by the user, for example, the volume parameter of the target client is set through the input device of the mobile terminal.
  • a target client volume control key may be set.
  • the target client volume control key includes at least two logical states, namely a first state and a second state, where the first state corresponds to the first parameter, The second state corresponds to the second parameter, and the audio of the target client can also be set to mute, that is, in the mute mode. In silent mode, the client can play all video files or audio files without sound.
  • the method for detecting whether the target client is in the silent mode is: determining whether other clients are currently playing audio; if there are other clients playing audio, it is determined that the target client is muted mode.
  • the system can check the startup and running data of the client installed in the mobile terminal through some specific API interfaces or SDK components.
  • the external interface of the audio manager of the system can be used to check whether any application in the current system is playing audio, for example, to obtain the return result of isMusicActive (), thereby determining whether other applications are playing audio files. If so, it is determined that the target client is in silent mode, otherwise, it is determined that the target client is not in silent mode.
  • S708 Control the audio and video output module of the electronic device to play the audio playback data and the video playback data at a silent volume.
  • the audio and video output module of the electronic device is controlled to play the audio playback data and the video playback data at a preset volume, where the preset volume may be the current system volume or the target The volume value set by the client.
  • an embodiment of the present application provides a video playback method, which is applied to the above electronic device, and the electronic device includes a system playback module and an FFmpeg module.
  • the method is an interaction process between the system playback module and the FFmpeg module ,
  • the method includes: S801 to S805.
  • the system playback module obtains the video file to be played sent by the target client.
  • the system playback module determines whether the video file to be played is supported.
  • the system playback module parses the video file to be played to obtain the audio stream and the video stream in the video file to be played, configures the identifier of the audio decoder for the audio stream, and configures the video decoder for the video stream Logo.
  • the system playback module sends the video stream, the audio stream, the identifier of the video decoder, and the identifier of the audio decoder to the FFmpeg module.
  • the FFmpeg module finds a corresponding video decoder according to the identifier of the video decoder and decodes the video stream to obtain video playback data, and finds a corresponding audio decoder according to the identifier of the audio decoder And decode the audio stream to obtain audio playback data.
  • the specific implementation of the FFmpeg module decoding the video file to be played to obtain audio playback data and video playback data may be the steps shown in FIG. 3 above.
  • the system playback module analyzes, the video stream, the The identifier of the audio stream, the video decoder, and the identifier of the audio decoder are sent to the FFmpeg module, so that the FFmpeg module finds the corresponding video decoder according to the identifier of the video decoder and decodes the video stream to Acquire video playback data, and find a corresponding audio decoder according to the identifier of the audio decoder and decode the audio stream to acquire audio playback data.
  • the FFmpeg module has the function of video parsing and decoding, and the specific implementation of the FFmpeg module to decode the video file to be played to obtain audio playback data and video playback data may be: the FFmpeg module obtains the to-be-played Audio stream and video stream in the video file; find the corresponding video decoder according to the type of the video stream and decode the video stream to obtain video playback data, and find the corresponding audio according to the type of the audio stream The decoder decodes the audio stream to obtain audio playback data.
  • the embodiment of the present application shows the interaction process of the system playback module and the FFmpeg module.
  • the interaction process of the system playback module and the FFmpeg module For a detailed implementation, refer to the foregoing embodiment.
  • FIG. 9 shows a structural block diagram of a video processing device 900 provided by an embodiment of the present application.
  • the device may include: an obtaining unit 901, a judging unit 902, a parsing unit 903, a configuration unit 904, and a processing unit 905.
  • the obtaining unit 901 is used to obtain the video file to be played sent by the target client.
  • the judging unit 902 is used to judge whether to support playing the video file to be played.
  • the parsing unit 903 is configured to parse the video file to obtain audio streams and video streams in the video file if it is not supported.
  • the configuration unit 904 is configured to configure the identifier of the audio decoder for the audio stream and the identifier of the video decoder for the video stream.
  • the processing unit 905 is configured to send the identification of the video stream, the audio stream, the video decoder, and the identification of the audio decoder to the FFmpeg module, so that the FFmpeg module searches according to the identification of the video decoder Go to the corresponding video decoder and decode the video stream to obtain video playback data, and find the corresponding audio decoder according to the identifier of the audio decoder and decode the audio stream to obtain audio playback data, wherein, The audio playback data and the video playback data can be played by the audio and video output module of the electronic device.
  • it also includes a playback unit for acquiring the audio playback data and video playback data returned by the FFmpeg module, and controlling the audio and video output module of the electronic device to play the audio playback data and video playback data.
  • the above units are located in the system playback module, which may be a program code for setting an embedded video playback module.
  • the coupling between the modules may be electrical, mechanical, or other forms of coupling.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules.
  • an embodiment of the present application further provides a video processing device 1000, including a system playback module 1001 and an FFmpeg module 1002.
  • the system play module 1001 is used to obtain the video file to be played sent by the target client; determine whether to play the video file to be played; if not, analyze the video file to obtain the audio stream in the video file And video streams, configure the audio decoder ID for the audio stream and the video decoder ID for the video stream, and send the video stream, the audio stream, the video decoder ID, and the audio decoder ID to The FFmpeg module 1002.
  • the FFmpeg module 1002 is used to find a corresponding video decoder according to the identifier of the video decoder and decode the video stream to obtain video playback data, and find a corresponding audio decoder according to the identifier of the audio decoder And decode the audio stream to obtain audio playback data, where the audio playback data and video playback data can be played by the audio and video output module of the electronic device.
  • the coupling between the modules may be electrical, mechanical, or other forms of coupling.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software function modules.
  • the electronic device 100 may be an electronic device capable of running an application program such as a smart phone, a tablet computer, an e-book.
  • the electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, an audio and video output module 30, and one or more application programs, where one or more application programs may be stored in the memory 120 It is configured to be executed by one or more processors 110, and one or more programs are configured to execute the method as described in the foregoing method embodiments.
  • the processor 110 may include one or more processing cores.
  • the processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, executes or executes instructions, programs, code sets or instruction sets stored in the memory 120, and calls data stored in the memory 120 to execute Various functions and processing data of the electronic device 100.
  • the processor 110 may use at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA)
  • DSP Digital Signal Processing
  • FPGA field programmable gate array
  • PROM programmable logic array
  • PLA programmable logic array
  • the processor 110 may integrate one or a combination of one of a central processing unit (Central Processing Unit, CPU), an image processing unit (Graphics Processing Unit, GPU), and a modem.
  • the CPU mainly handles the operating system, user interface and application programs, etc .
  • the GPU is used for rendering and rendering of the displayed content
  • the modem is used for processing wireless communication. It can be understood that the above-mentioned modem may not be integrated into the processor 110, and is implemented by a communication chip alone.
  • the processor 110 includes a system playback module 1001 and an FFmpeg module 1002.
  • a system playback module 1001 and FFmpeg module 1002 For specific implementations of the system playback module 1001 and FFmpeg module 1002, reference may be made to the foregoing embodiments, and details are not described herein again.
  • the memory 120 may include a random access memory (Random Access Memory, RAM) or a read-only memory (Read-Only Memory).
  • the memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets.
  • the memory 120 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, sound playback function, image playback function, etc.) , Instructions for implementing the following method embodiments.
  • the storage data area may also store data created by the electronic device 100 in use (such as a phone book, audio and video data, and chat history data).
  • FIG. 12 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • the computer-readable storage medium 1200 stores program codes, and the program codes can be called by a processor to execute the method described in the foregoing method embodiments.
  • the computer-readable storage medium 1200 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 1200 includes a non-transitory computer-readable storage medium.
  • the computer-readable storage medium 1200 has a storage space for the program code 1210 to perform any of the method steps described above. These program codes can be read from or written into one or more computer program products.
  • the program code 1210 may be compressed in an appropriate form, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请公开了一种视频处理方法、装置、电子设备及计算机可读介质，涉及视频处理技术领域。该方法包括：***播放模块获取目标客户端发送的待播放视频文件；判断是否支持播放待播放视频文件；若不支持，将待播放视频文件解析以获得待播放视频文件内的音频流和视频流；为音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；将视频流、音频流、视频解码器的标识以及音频解码器的标识发送至FFmpeg模块，以使FFmpeg模块将待播放视频文件解码以获取音频播放数据和视频播放数据，其中，音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。因此，通过调用FFmpeg模块，能够使得客户端将不支持播放的视频文件顺利播放。

Description

视频处理方法、装置、电子设备及计算机可读介质
相关申请的交叉引用
本申请要求于2018年10月15日提交中国专利局的申请号为CN201811198856.9、名称为“视频处理方法、装置、电子设备及计算机可读介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，更具体地，涉及一种视频处理方法、装置、电子设备及计算机可读介质。
背景技术
随着电子技术和信息技术的发展，越来越多的设备能够播放视频。安装在设备上的客户端在视频播放的过程中，需要对视频执行解析、解码、渲染以及合成等操作，再在显示屏上显示视频流并通过音频设备播放音频流。但是，如果有些客户端不支持某个视频的播放，会导致用户体验过差。
发明内容
本申请提出了一种视频处理方法、装置、电子设备及计算机可读介质，以改善上述缺陷。
第一方面，本申请实施例提供了一种视频处理方法，应用于电子设备，所述电子设备包括***播放模块以及FFmpeg模块。该方法包括：所述***播放模块获取目标客户端发送的待播放视频文件；判断是否支持播放所述待播放视频文件；若不支持，将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流；为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
第二方面，本申请实施例还提供了一种视频处理方法，应用于电子设备，所述电子设备包括***播放模块以及FFmpeg模块。该方法包括：所述***播放模块获取目标客户端发送的待播放视频文件；所述***播放模块判断是否支持播放所述待播放视频文件；若不支持，所述***播放模块将 所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；所述***播放模块将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块；所述FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
第三方面，本申请实施例还提供了一种视频处理装置，包括：获取单元、判断单元、解析单元、配置单元和处理单元。获取单元，用于获取目标客户端发送的待播放视频文件。判断单元，用于判断是否支持播放所述待播放视频文件。解析单元，用于若不支持，将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流。配置单元，用于为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识。处理单元，用于将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
第四方面，本申请实施例还提供了一种视频处理装置，包括***播放模块以及FFmpeg模块。所述***播放模块用于获取目标客户端发送的待播放视频文件；获取所述待播放视频文件的目标类型；判断是否支持播放所述目标类型的视频文件；若不支持，将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识，将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块。所述FFmpeg模块用于根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
第五方面，本申请实施例还提供了一种电子设备，包括：图像处理器、存储器、***播放模块以及FFmpeg模块和一个或多个应用程序。其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述图像处理器执行，所述一个或多个程序配置用于执行上述方法。
第六方面，本申请实施例还提供了一种计算机可读取存储介质，计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行上述方法。
相对于现有技术，本申请提供的方案，在客户端播放待播放视频文件时，调用***播放模块播放该视频文件，则***播放模块获取到该待播放视频文件，根据该待播放视频文件的类型确定是否支持播放该类型的视频文件，如果不支持，则将待播放视频文件解析为视频流和音频流，并分别配置对应的解码器标识之后一起发送至FFmpeg模块。FFmpeg模块将所述音频流和视频流根据各自对应的解码器解码之后以获取音频播放数据和视频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放，则电子设备的音视频***能够播放音频播放数据和视频播放数据。因此，通过调用FFmpeg模块，能够将不支持播放的视频文件顺利播放。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简 单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例提供的视频播放架构的模块框图；
图2示出了本申请一实施例提供的视频处理方法的方法流程图；
图3示出了图1中的S204的方法流程图；
图4示出了本申请另一实施例提供的视频处理方法的方法流程图；
图5示出了本申请一实施例提供的客户端播放视频时界面变化示意图；
图6示出了本申请另一实施例提供的客户端播放视频时界面变化示意图。
图7示出了本申请又一实施例提供的视频处理方法的方法流程图；
图8示出了本申请再又一实施例提供的视频处理方法的方法流程图；
图9示出了本申请一实施例提供的视频处理装置的模块框图；
图10示出了本申请另一实施例提供的视频处理装置的模块框图；
图11示出了本申请实施例提供的电子设备的结构框图；
图12示出了本申请实施例的用于保存或者携带实现根据本申请实施例的视频处理方法的程序代码的存储单元。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
请参阅图1，示出了视频播放架构的框图。如图1所示，该***包括客户端10、***播放模块20和音视频输出模块30。
客户端10在获取到待播放视频文件时，将该视频文件发送至***播放模块20，***播放模块20在获取到待播放的数据的时候，接下来就是解析音视频数据了。一般的视频文件都有视频流和音频流两部分组成，不同的视频格式音视频的封装格式肯定不一样。将音频流和视频流合成文件的过程称为muxer,反之从媒体文件中分离音频流和视频流的过程称为demuxer。其中，***播放模块20为电子设备的操作***内默认的播放模块，即在电子设备内安装的客户端播放视频文件时，默认为调用该***播放模块20对视频文件播放。于安卓***内，该***播放模块20可以是Media Framework模块。
播放视频文件就需要从文件流中分离出音频流和视频流，即对视频文件解析的过程，该过程由***播放模块20内的解析子模块201，其中，该解析子模块201可以是安卓***内的MediaExtractor模块，另外，解析的过程除了将视频文件解析成视频流和音频流，还需要为视频流和音频流配置对应的解码器，即配置对应的解码策略。解码后的视频帧可以直接渲染,音频帧可以送到音频输出设备的缓冲区进行播放,当然,视频渲染和音频播放的时间戳一定要控制同步，其中，解码的操作由***播放模块20内的解码子模块202执行。其中，该解码子模块202可以是安卓***内的MediaCodec。
具体地，视频解码可以包括硬解码和软解码，硬件解码是将原来全部交由中央处理器(Central Processing Unit，CPU)来处理的视频数据的一部分交由图像处理器(Graphics Processing Unit，GPU)来做，而GPU的并行运算能力要远远高于CPU，这样可以大大的降低对CPU的负载，CPU的占用率较低了之后就可以同时运行一些其他的程序了，当然，对于较好的处理器来说，比如i5 2320， 或者AMD任何一款四核心处理器来说，硬解和软件依据需求而选定。
解码之后的音频流变为音频播放数据，例如，可以是PCM格式的音频数据，则音视频输出模块30根据PCM编码驱动音频播放设备的音圈等部件振动发声。同理，解码之后的视频流变为视频播放数据，例如，可以是YUV格式的视频数据，音视频输出模块30将YUV格式的视频数据发送至图层传递模块(SurfaceFlinger)，由SurfaceFlinger将解码后的视频数据渲染和合成之后，在显示屏上显示。其中，SurfaceFlinger是一个独立的Service，它接收所有Window的Surface作为输入，根据ZOrder、透明度、大小、位置等参数，计算出每个Surface在最终合成图像中的位置，然后交由HWComposer或OpenGL生成最终的显示Buffer,然后显示到特定的显示设备上。
但是，发明人在研究中发现，由于***更新问题或者客户端版本问题，该***播放模块对于一些格式的视频文件不支持播放，例如，MP4格式，如果不支持该格式的视频文件的播放，会调用第三方应用播放，也就是说，在使用***播放模块的时候，无法对某个视频文件解码，则可以调用第三方应用对该视频文件软解码，从而获取能够被音视频输出模块播放的音频和视频数据。但是，使用第三方应用播放该视频文件，会将第三方应用调起，即当前的客户端被切换至后台而第三方应用被调起至前台，则会打断用户针对客户端的操作，并且，还会导致无法实现客户端对视频的静音播放功能。
因此，为了克服上述缺陷，请参阅图2，本申请实施例提供了一种视频处理方法，应用于上述电子设备，所述电子设备包括***播放模块以及FFmpeg模块，具体地，该方法的执行主体是***播放模块，则该方法包括：S201至S204。
S201：所述***播放模块获取目标客户端发送的待播放视频文件。
具体地，客户端为安装在电子设备内的应用程序，具有视频播放的功能。客户端在***桌面设有图标，用户点击该客户端的图标，能够将该客户端打开，例如，从用户点击的应用的包名来确认，视频应用的包名可以***后台从代码中获取，包名格式为：com.android.video。
客户端的视频列表界面内显示有多个视频对应的显示内容，多个视频对应的显示内容包括每个视频对应的缩略图，该缩略图可以作为一个触摸按键使用，用户点击该缩略图，客户端能够检测到用户所想点击的缩略图，也就能够确定欲播放的待播放视频文件。
客户端响应用户在视频列表内选中的视频，进入视频的播放界面，点击该播放界面的播放按钮，客户端通过对用户触控操作的监听，能够检测到用户当前所点击的是什么类型的视频，具体地，播放按钮设置有预设属性，检测到所获取的触控操作对应的播放按钮的属性，就能够确定用户所选中的待播放视频。
客户端在获取到用户输入的待播放视频文件之后，将该待播放视频文件发送至***播放模块，以便***播放模块对该视频文件解析解码之后播放。
S202：判断是否支持播放所述待播放视频文件。
***播放模块对所支持的视频文件能够解析成视频流和音频流，并且能够成功将视频流和音频流解码，获得到音频播放数据和视频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放，例如，该音频播放数据和视频播放数据分别为音频PCM数据及视频YUV数据。
而对于***播放模块不支持播放的视频文件，该***播放模块无法对该视频文件解析，即无法由该视频文件获取到对应的视频流和音频流，或者，即使可以是对视频文件正常解析，即能够由该视频文件获取到对应的视频流和音频流，而由于无法对该格式的视频流和音频流解码，也就无法获 得到音频播放数据和视频播放数据，而无法正常播放该视频文件。
具体地，获取所述待播放视频文件的目标类型。
作为一种实施方式，可以通过视频文件的扩展名判断该视频文件的类型，例如，扩展名是.mp4,则该待播放视频文件的目标类型是mp4。
判断是否支持播放所述目标类型的待播放视频文件。其中，视频文件的类型可以是视频文件的格式。则***播放模块对应一个支持播放的视频文件的格式列表，在该格式列表内包括多个格式的标识，且该格式列表内的每个格式对应的视频文件均能够被正常播放，具体地，该格式列表内可以包括视频文件的格式对应的音频流的格式以及视频流的格式，以及每个视频流的格式和和音频流的格式对应的解码器的标识，通过查找该格式列表，就能够正常为所支持的视频格式配置正确的解码器。其中，视频文件的格式即为视频文件的类型，而视频流的格式为视频流的类型，而音频流的格式为音频流的类型。
因此，在获取到视频文件的目标类型之后，通过对该格式列表的查找，判断该视频文件的目标类型与该格式列表内的视频文件的类型是否匹配，如果存在匹配的类型，则判定支持播放所述目标类型的视频文件，如果不存在匹配的类型，则判定不支持播放所述目标类型的视频文件。
S203：将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流。
***播放模块利用解析子模块解析该待播放视频文件，以获取对应的视频流和音频流，例如，可以是通过上述所提及的demuxer过程，能够对待播放视频文件解封装之后，获取到待播放视频文件对应的视频流和音频流。
S204：为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识。
具体地，如图3所示，S204的步骤可以包括：S2041至S2043。
S2041：确定所述视频流的类型和所述音频流的类型。
同理，在获取到视频流和音频流之后，也可以通过获取该视频流和音频流的扩展名而确定该视频流的类型和所述音频流的类型。
S2042：根据所述视频流的类型为所述视频流配置视频解码器的标识。
预先获取有视频流的类型与视频解码器的标识的第一对应关系，该第一对应关系可以由用户手动输入，例如，根据目前已经确定能够解码该类型的视频文件的视频解码器的介绍而输入该对应关系内，例如，由一个网络服务器内获取该第一对应关系，而该网路服务器内统计了每种类型的视频流能够被解码的解码器的标识。
当然，也可以是由FFmpeg模块获取到该第一对应关系，具体地，可以是在安装该FFmpeg模块的时候，由FFmpeg模块对应的格式列表内对应的每个类型的视频流对应的解码器的标识而设定该第一对应关系。
则在获取到待播放视频文件对应的视频流的类型之后，由该第一对应关系查找能够解码该类型的视频流的解码器的标识，从而为所述视频流配置视频解码器的标识。
S2043：根据所述音频流的类型为所述音频流配置音频解码器的标识。
同理，也预先获取有音频流的类型与音频解码器的标识的第二对应关系，且该第二对应关系也可以采用诸如上述第一对应关系的获取方式。则在获取到待播放视频文件对应的音频流的类型之后，由该第二对应关系查找能够解码该类型的音频流的音频解码器的标识，从而为所述音频流配置音频解码器的标识。
S205：将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg 模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
其中，FFmpeg模块是一个开源的跨平台的视频和音频流框架，属于自由软件，采用LGPL或GPL许可证(依据选择的组件)。它提供了录制、转换以及流化音视频的完整解决方案。它包含了丰富的音频/视频编解码库libavcodec。
FFmpeg模块由以下几部分组成:
1.ffmpeg视频文件转换命令行工具。
2.ffserver基于HTTP、RTSP用于实时广播的多媒体服务器。
3.ffplay用SDL和FFmpeg库开发的一个简单的媒体播放器。
4.libavcodec一个包含了所有FFmpeg音视频编解码器的库。
5.libavformat一个包含了所有的普通音视格式的解析器和产生器的库。
因此，FFmpeg模块集成了多种格式的音频和视频的编码器，并且具有音视频播放功能，还具有视频文件的解析功能。
则将FFmpeg模块注入电子设备的操作***内，并且定义好***播放模块与FFmpeg模块之间的调用接口，例如，可以是一个API也可以是一个SDK，从而***播放模块与FFmpeg模块之间能够通信。
则在***播放模块判定不支持客户端请求播放的视频文件的正常播放时，将该视频文件发送至FFmpeg模块，而由于FFmpeg模块的强大的编解码能力，能够支持大量格式的视频文件的解析和解码。并且，FFmpeg模块能够植入到操作***内，且FFmpeg模块可以作为一个服务进程而在***的后台内对视频文件解码，从而生成音频播放数据和视频播放数据，而不必切换至前台，而使得客户端被迫切换至后台，而避免打断用户对客户端的操作。
另外，在FFmpeg模块获取到音频播放数据和视频播放数据，可以将音频播放数据和视频播放数据发送至***播放模块，而***播放模块控制电子设备的音视频输出模块，利用音频播放设备(例如，扬声器、听筒或者耳机等)播放音频播放数据，并且利用视频播放设备(例如，屏幕)播放视频播放数据。
当然，也可以是FFmpeg模块直接控制电子设备的音视频输出模块输出音频播放数据和视频播放数据。
则由于FFmpeg模块具有视频文件的解析和解码能力，***播放模块可以将待播放视频文件直接交给FFmpeg模块，则FFmpeg模块对该待播放视频文件解析和解码，另外，也可以由***播放模块对该待播放视频文件解析之后获得到视频流和音频流之后再发送给FFmpeg模块进行解码。
***播放模块在获取到视频流与该视频流对应的视频解码器的标识，以及音频流与该音频流对应的音频解码器的标识之后，一起打包发送至FFmpeg模块，具体地，视频流和音频流存储在电子设备的存储空间内，并且定义了句柄或指针等地址标识，而将视频流与音频流发送至FFmpeg模块的过程为，将该音频流的地址标识与视频流的地址标识发送至FFmpeg模块，则FFmpeg模块根据该音频流的地址标识能够由电子设备的存储空间内获取该地址标识对应的音频流，同理，也可以根据该视频流的地址标识由电子设备的存储空间内获取该地址标识对应的视频流。
FFmpeg模块根据预先存储的各个解码器的标识与解码器的对应关系，确定视频解码器的标识对 应的视频解码器以及音频解码器的标识对应的音频解码器。然后，再调用该视频解码器对视频流解码，以获取视频播放数据，以及调用该音频解码器对音频流解码，以获取音频播放数据。从而，通过调用FFmpeg模块，能够将那些不支持播放的视频文件顺利播放。
下面，以目标类型为MP4类型为例，说明本申请的方法，如图4所示，该视频处理方法包括：S401至S413。
S401：通过MediaEtractor获取待播放视频文件解析。
电子设备内安装的客户端在播放视频的时候，通过与MediaEtractor的API接口将待播放视频文件发送至***播放模块，即在客户端的默认播放流程下，客户端会将待播放视频数据发送至MediaEtractor进行解析，则MediaEtractor与客户端之间设置有API接口。
S402：将待播放视频文件发送至MPEG4Extractor。
具体地，在MediaEtractor获取到待播放视频文件之后，确定该待播放视频文件为MP4类型，则选择采用MPEG4Extractor对该MP4类型视频文件解析，则将待播放视频文件发送至MPEG4Extractor，其中，MediaEtractor和MPEG4Extractor均为植入到***播放模块内的程序模块。
S403：判断是否支持播放MP4格式的视频文件。
然后，***播放模块判断是否支持播放MP4格式的视频文件，具体地，可以是MPEG4Extractor判断是否支持播放MP4格式的视频文件。
S404：原生解析逻辑。
如果支持MP4格式的视频文件的播放，则可以按照原生解析逻辑解析该视频文件。具体地，可以是采用MediaExtractor解析视频数据，从而得到视频流和音频流，再采用MediaCodec解码视频流和音频流，得到音频PCM数据及视频YUV数据处理之后，再分别送到音、视频输出***输出。
S405：通过MPEG4Extractor对待播放视频文件解析。
如果不支持MP4格式的视频文件的播放，则采用MPEG4Extractor对待播放视频文件解析，具体地，确定视频文件的类型为MP4格式，为该待播放视频文件配置类型ID，例如，类型ID为mimetype。
其中，类型ID用于表示该视频文件的类型，于本申请实施例中，为MP4格式定义的类型ID与***播放模块内所记录的类型ID不一致，由此，能够避免***播放模块将该类型ID对应的视频文件采用原生解析逻辑解析而发生错误，因为解析的过程，除了将视频文件解封装成视频流和音频流之外，还需要为为视频流配置视频解码器以及为音频流配置音频解码器，而如果采用原生解析逻辑解析时，即使将获得到视频流和音频流也无法成功配置对应的解码器，而发生错误，而在***播放模块无法识别本申请所定义的类型ID，即mimetype，不会对该类型ID进行解析和解码操作。则针对类型ID为mimetype能够采用本申请实施例所对应的解析和解码逻辑解析，而不采用原生解析逻辑解析。
S406：配置音频流为audio/ffmpeg。
具体地，在上述类型ID为mimetype的基础上，为音频流配置身份标识audio/ffmpeg，该身份标识作为该音频流的身份标识，便于在配置解码器的时候，在确定该音频流的身份标识为audio/ffmpeg时，选择采用本申请的解析和解码方法，即利用FFmpeg模块的视频处理方法，也就是说，该音频流的身份标识与解码策略对应。
S407：判断音频流格式。
其中，针对MP4格式的视频文件，音频流格式包括mp3和adpcm，则判断该音频流格式为mp3还是adpcm，如果是mp3，则执行S408，如果是adpcm，则执行S409。
S408：配置音频流的音频解码器的标识为FFMPEG_CODEC_ID_MP3。
具体地，可以在MPEG4Extractor内添加解析MP4格式的程序代码，例如，添加解析这类chunk type box的代码，使得MPEG4Extractor能够为MP4格式的视频文件通过所添加的代码对其解析以及配置解码器。
具体地，如果音频流为MP3，则配置的音频解码器的标识为FFMPEG_CODEC_ID_MP3，该标识对应的解码器能够解码MP3格式的音频流。例如，解析及配置解码器ID如下：
#define#define FFMPEG_CODEC_ID_MP3 0x15001。
音频流mp3:Chunk type为.mp3，配置音频流MimeType为audio/ffmpeg,CodecID为FFMPEG_CODEC_ID_MP3。
其中，FFMPEG_CODEC_ID_MP3为音频解码器的标识，为该音频解码器的标识赋值，即赋值0x15001，则将音频解码器的标识发送至FFmpeg的时候，将FFMPEG_CODEC_ID_MP3以及所赋的值0x15001一起发送至FFmpeg，则FFmpeg可以通过FFMPEG_CODEC_ID_MP3查找到对应的音频解码器为MP3解码器，也可以根据0x15001查找到对应的解码器。
例如，FFmpeg内4类编码器定义值：
AV_CODEC_ID_ADPCM_MS＝0x11006；
AV_CODEC_ID_MJPEG＝0x8；
AV_CODEC_ID_MP3＝0x15001；
AV_CODEC_ID_SVQ3＝0x18。
则在***播放模块的MPEG4Extractor内，为MP3格式的音频流配置的音频解码器的赋值与在FFmpeg内MP3格式的音频解码器的赋值相同，例如，都是0X15001，则通过0X15001能够快速查找到该MP3音频解码器。当然，通过分析音频解码器的标识，例如，FFMPEG_CODEC_ID_MP3，也能够确定该标识对应的音频解码器也是MP3音频解码器，例如，通过标识的关键字提取，可以提取出“MP3”的文本，就能够确定该标识对应的是MP3音频解码器。
S409：配置音频流的音频解码器的标识为FFMPEG_CODEC_ID_ADPCM_MS。
同理，如果音频流的格式为adpcm，则配置的音频流的音频解码器的标识为FFMPEG_CODEC_ID_ADPCM_MS，该标识对应的解码器能够解码adpcm格式的音频流。例如，解析及配置解码器ID如下：
#define FFMPEG_CODEC_ID_ADPCM_MS 0x11006。
音频流adpcm:Chumk type为ms02,配置音频流MimeType为audio/ffmpeg,CodecID为FFMPEG_CODEC_ID_ADPCM_MS。
其中，FFMPEG_CODEC_ID_ADPCM_MS为音频解码器的标识，为该音频解码器的标识赋值，即赋值0x11006，则将音频解码器的标识发送至FFmpeg的时候，将FFMPEG_CODEC_ID_ADPCM_MS以及所赋的值0x11006一起发送至FFmpeg，则FFmpeg可以通过FFMPEG_CODEC_ID_ADPCM_MS查找到对应的音频解码器为adpcm解码器，也可以根据0x11006查找到对应的解码器，具体过程同上述MP3的过程，在此不再赘述。
S410：配置视频流为video/ffmpeg。
具体地，在上述类型ID为mimetype的基础上，为视频流配置身份标识video/ffmpeg，该身份标识作为该视频流的身份标识，便于在配置解码器的时候，在确定该视频流的身份标识为video/ffmpeg时，选择采用本申请的解析和解码方法，即利用FFmpeg模块的视频处理方法，也就是 说，该视频流的身份标识与解码策略对应。
S411：判断视频流格式。
其中，针对MP4格式的视频文件，视频流格式包括mjpeg和sorenson，则判断该音频流格式为mjpeg还是sorenson，如果是mjpeg，则执行S412，如果是sorenson，则执行S413。
S412：配置视频流的视频解码器的标识为FFMPEG_CODEC_ID_MJPEG。
同理，如果视频流的格式为mjpeg，则配置的视频流的视频解码器的标识为FFMPEG_CODEC_ID_MJPEG，该标识对应的解码器能够解码mjpeg格式的视频流。例如，解析及配置解码器ID如下：
#define FFMPEG_CODEC_ID_MJPEG 0x8。
视频流mjpeg:Chunk type为jpeg,配置视频流MimeType为video/ffmpeg,CodecID为FFMPEG_CODEC_ID_MJPEG。
其中，FFMPEG_CODEC_ID_MJPEG为视频解码器的标识，为该视频解码器的标识赋值，即赋值0x8，则将视频解码器的标识发送至FFmpeg的时候，将FFMPEG_CODEC_ID_MJPEG以及所赋的值0x8一起发送至FFmpeg，则FFmpeg可以通过FFMPEG_CODEC_ID_MJPEG查找到对应的视频解码器为mjpeg解码器，也可以根据0x8查找到对应的解码器，具体过程同上述MP3的过程，在此不再赘述。
S413：配置视频流的视频解码器的标识为FFMPEG_CODEC_ID_SORENSON。
同理，如果视频流的格式为sorenson，则配置的视频流的视频解码器的标识为FFMPEG_CODEC_ID_SORENSON，该标识对应的解码器能够解码sorenson格式的视频流。例如，解析及配置解码器ID如下：
#define FFMPEG_CODEC_ID_SORENSON 0x18。
视频流sorenson:Chunk type为sorenson，配置视频流MimeType为video/ffmpeg,CodecID为FFMPEG_CODEC_ID_SORENSON。
其中，FFMPEG_CODEC_ID_SORENSON为视频解码器的标识，为该视频解码器的标识赋值，即赋值0x18，则将视频解码器的标识发送至FFmpeg的时候，将FFMPEG_CODEC_ID_SORENSON以及所赋的值0x18一起发送至FFmpeg，则FFmpeg可以通过FFMPEG_CODEC_ID_SORENSON查找到对应的视频解码器为sorenson解码器，也可以根据0x18查找到对应的解码器，具体过程同上述MP3的过程，在此不再赘述。
通过上述MP4解析器的扩展，对于之前这类不支持的视频可以正常解析，得到需要解码的二进制数据。
APP调用MediaCodec并传出正确的音视频流MimeType，选择***中集成的ffmpeg解码器***，并且通过CodecID精确配置ffmpeg中的相关解码器。
另外，客户端在播放***播放模块不支持的视频文件的时候，相比通过调起第三方应用软解码该视频文件以播放视频，采用本申请的方法能够避免切换至第三方应用的视频播放界面，而是利用***播放模块播放视频文件，不必进行跳转。
具体地，如图5所示，图5(a)的画面为客户端的界面，在该界面内显示有待播放视频文件的播放按钮，即“视频内容”对应的播放按钮，用户点击该播放按钮，如果调起第三方应用播放该视频，则客户端会被切换至后台，而第三方应用被调起并且被切换至前台，如图5(b)所示。而如果采用本申请的视频处理方法，则***播放模块和FFmpeg在后台对视频解析和解码，则不会导致客户端被切换至后台，如图6(b)所示，依然在客户端的界面上播放该视频。
另外，客户端不调起第三方应用程序播放该视频，也能够避免用户在对客户端静音之后，由于使用第三方应用程序播放该视频，却依然导致视频播放发出声音。具体地，如图7所示，本申请实施例提供了一种视频处理方法，应用于上述电子设备，所述电子设备包括***播放模块以及FFmpeg模块，具体地，该方法的执行主体是***播放模块，则该方法包括：S701至S708。
S701：所述***播放模块获取目标客户端发送的待播放视频文件。
S702：判断是否支持播放所述待播放视频文件。
S703：将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流。
S704：为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识。
S705：将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
S706：获取所述FFmpeg模块返回的所述音频播放数据和视频播放数据。
在FFmpeg模块获取到音频播放数据和视频播放数据，可以将音频播放数据和视频播放数据发送至***播放模块，而***播放模块控制电子设备的音视频输出模块，利用音频播放设备(例如，扬声器、听筒或者耳机等)播放音频播放数据，并且利用视频播放设备(例如，屏幕)播放视频播放数据。
当然，也可以是FFmpeg模块直接控制电子设备的音视频输出模块输出音频播放数据和视频播放数据。
于本申请实施例中，FFmpeg模块将解码之后的音频播放数据和视频播放数据返回至***播放模块，则***播放模块能够控制所述电子设备的音视频输出模块播放所述音频播放数据和视频播放数据，由此，***播放模块继续使用原生播放逻辑播放视频，可移植性更高。
S707：检测所述目标客户端是否处于静音模式。
其中，目标客户端为请求播放待播放视频文件的客户端。
作为一种实施方式，确认所述目标客户端是否处于静音模式的方式为：获取目标客户端的音量参数，如果所述音量参数为目标参数，则判定目标客户端处于静音模式。如果所述音量参数不是目标参数，则判定目标客户端不处于静音模式。
于本申请实施例中，第一参数与静音状态对应，第二参数与非静音状态对应，该目标参数为第一参数。目标客户端在播放音频时，读取与该目标客户端对应的音量参数，如果所读取的音量参数为第一参数，即所述音量参数为目标参数，则将目标客户端静音，即此时应用程序的音量状态为静音状态。如果音量参数是第二参数，即所述音量参数不是第一参数，则指示所述目标客户端按照预设音量播放所述视频文件，即此时应用程序的音量状态为非静音状态。其中，将目标客户端静音的具体实施方式可以是，禁止播放目标客户端内的视频文件，也可以是，以静音方式播放所述视频文件，即运行该视频文件但是不通过扬声器或耳机等设备播放该视频文件。
具体地，目标客户端的音量参数可以是由用户设定的，例如，通过移动终端的输入设备为该目标客户端设定音量参数。作为一种实施方式，可以设置一个目标客户端音量控制键，该目标客户端音量控制键至少包括两个逻辑状态，分别为第一状态和第二状态，其中，第一状态对应第一参数，第二状态对应第二参数，而且，也能够将目标客户端的音频设置为静音，即处于静音模式。在静音模式下，客户端可以是播放所有的视频文件或者音频文件均不发出声音。
作为另一种实施方式，检测所述目标客户端是否处于静音模式的方式为：判断当前是否存在其他客户端正在播放音频；若存在其他客户端正在播放音频，则判定所述目标客户端处于静音模式。
判断当前是否存在其他客户端正在播放音频，如果存在其他客户端正在播放音频，则判定所述网页客户端满足静音条件，如果不存在其他客户端正在播放音频，则判定所述网页客户端不满足静音条件。具体地，***可以通过一些特定的API接口或者SDK组件来查阅所安装在移动终端内的客户端的启动、运行数据等情况。具体地，可以通过***的音频管理器的对外接口来查看当前***内是否有应用正在播放音频，例如，获取isMusicActive()的返回结果，由此确定是否有其他应用正在播放音频文件。如果有，则判定目标客户端处于静音模式，否则，则判定目标客户端未处于静音模式。
S708：控制所述电子设备的音视频输出模块以静音音量播放所述音频播放数据和视频播放数据。
而如果目标客户端未处于静音模式，则控制电子设备的音视频输出模块以预设音量播放所述音频播放数据和视频播放数据，其中，预设音量可以是当前的***音量，也可以是目标客户端被设置的音量值。
请参阅图8，本申请实施例提供了一种视频播放方法，应用于上述电子设备，所述电子设备包括***播放模块以及FFmpeg模块，具体地，该方法为***播放模块与FFmpeg模块的交互过程，则该方法包括：S801至S805。
S801：所述***播放模块获取目标客户端发送的待播放视频文件。
S802：所述***播放模块判断是否支持播放所述待播放视频文件。
S803：所述***播放模块将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识。
S804：所述***播放模块将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块。
S805：所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
其中，FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据的具体实施方式，可以是上述图3所示的步骤，***播放模块解析之后，将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
作为另一种实施方式，FFmpeg模块具有视频解析解码的功能，则FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据的具体实施方式可以是：FFmpeg模块获取所述待播放视频文件内的音频流和视频流；根据所述视频流的类型查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频流的类型查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
具体地，本申请实施例示出了***播放模块与FFmpeg模块的交互过程，详细的实施方式可以参考前述实施例。
请参阅图9，其示出了本申请实施例提供的一种视频处理装置900的结构框图，该装置可以包括：获取单元901、判断单元902、解析单元903、配置单元904和处理单元905。
获取单元901，用于获取目标客户端发送的待播放视频文件。
判断单元902，用于判断是否支持播放所述待播放视频文件。
解析单元903，用于若不支持，将所述视频文件解析以获得所述视频文件内的音频流和视频流。
配置单元904，用于为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识。
处理单元905，用于将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
另外，还包括播放单元，用于获取所述FFmpeg模块返回的所述音频播放数据和视频播放数据，控制所述电子设备的音视频输出模块播放所述音频播放数据和视频播放数据。
其中，上述各单位位于***播放模块内，可以是设置植入视频播放模块的程序代码。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
请参阅图10，本申请实施例还提供了一种视频处理装置1000，包括***播放模块1001以及FFmpeg模块1002。
所述***播放模块1001用于获取目标客户端发送的待播放视频文件；判断是否支持播放所述待播放视频文件；若不支持，将所述视频文件解析以获得所述视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识，将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块1002。
所述FFmpeg模块1002用于根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
请参考图11，其示出了本申请实施例提供的一种电子设备的结构框图。该电子设备100可以是智能手机、平板电脑、电子书等能够运行应用程序的电子设备。本申请中的电子设备100可以包括一个或多个如下部件：处理器110、存储器120、音视频输出模块30以及一个或多个应用程序，其中一个或多个应用程序可以被存储在存储器120中并被配置为由一个或多个处理器110执行，一个或多个程序配置用于执行如前述方法实施例所描述的方法。
处理器110可以包括一个或者多个处理核。处理器110利用各种接口和线路连接整个电子设备100内的各个部分，通过运行或执行存储在存储器120内的指令、程序、代码集或指令集，以及调用存储在存储器120内的数据，执行电子设备100的各种功能和处理数据。可选地，处理器110可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器110可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作***、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器110中，单独通过一块通信芯片进行实现。
另外，处理器110包括***播放模块1001和FFmpeg模块1002，则***播放模块1001和FFmpeg模块1002的具体实施方式可参考前述实施例，在此不再赘述。
存储器120可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器120可用于存储指令、程序、代码、代码集或指令集。存储器120可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作***的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储电子设备100在使用中所创建的数据(比如电话本、音视频数据、聊天记录数据)等。
请参考图12，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读存储介质1200中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质1200可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质1200包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质1200具有执行上述方法中的任何方法步骤的程序代码1210的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码1210可以例如以适当形式进行压缩。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种视频处理方法，其特征在于，应用于电子设备，所述电子设备包括***播放模块以及FFmpeg模块，所述方法包括：
    所述***播放模块获取目标客户端发送的待播放视频文件；
    判断是否支持播放所述待播放视频文件；
    若不支持，将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流；为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；
    将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
  2. 根据权利要求1所述的方法，其特征在于，所述为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识，包括：
    确定所述视频流的类型和所述音频流的类型；
    根据所述视频流的类型为所述视频流配置视频解码器的标识；
    根据所述音频流的类型为所述音频流配置音频解码器的标识。
  3. 根据权利要求1所述的方法，其特征在于，所述判断是否支持播放所述待播放视频文件，包括：
    获取所述待播放视频文件的目标类型；
    判断是否支持播放所述目标类型的待播放视频文件。
  4. 根据权利要求3所述的方法，其特征在于，所述判断是否支持播放所述目标类型的待播放视频文件，包括：
    判断在格式列表内是否存在与所述待播放视频文件的目标类型匹配的类型，所述格式列表包括至少一个支持播放的视频文件的类型；
    若存在匹配的类型，判定支持播放所述目标类型的待播放视频文件；
    若不存在匹配的类型，判定不支持播放所述目标类型的待播放视频文件。
  5. 根据权利要求1所述的方法，其特征在于，所述若不支持，将所述待播放视频文件解析以获得所述待播放视频文件内的音频流和视频流，包括：
    若不支持，将所述待播放视频文件发送至FFmpeg模块，以使所述FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据。
  6. 根据权利要求3-5任一项所述的方法，其特征在于，所述***播放模块为MediaPlayer模块，所述目标类型为MP4类型。
  7. 根据权利要求1或2所述的方法，其特征在于，所述将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块之后，还包括：
    获取所述FFmpeg模块返回的所述音频播放数据和视频播放数据；
    控制所述电子设备的音视频输出模块播放所述音频播放数据和视频播放数据。
  8. 根据权利要求7所述的方法，其特征在于，所述控制所述电子设备的音视频输出模块播放所 述音频播放数据和视频播放数据，包括：
    检测所述目标客户端是否处于静音模式；
    若处于静音模式，则控制所述电子设备的音视频输出模块以静音音量播放所述音频播放数据和视频播放数据。
  9. 根据权利要求8所述的方法，其特征在于，所述检测所述目标客户端是否处于静音模式，包括：
    判断当前是否存在其他客户端正在播放音频；
    若存在其他客户端正在播放音频，则判定所述目标客户端处于静音模式。
  10. 根据权利要求8所述的方法，其特征在于，所述检测所述目标客户端是否处于静音模式，包括:
    获取所述目标客户端的音量参数；
    若所述音量参数为目标参数，则判定所述目标客户端处于静音模式；
    若所述音量参数不为目标参数，则判定目标客户端处于非静音模式。
  11. 根据权利要求10所述的方法，其特征在于，所述音量参数包括第一参数和第二参数，所述第一参数与所述静音模式对应，所述第二参数与所述非静音模式对应，所述第一参数为所述目标参数，所述方法还包括：
    获取所述目标客户端的音量参数；若所述音量参数为目标参数，则将所述目标客户端静音；
    若所述音量参数不为目标参数，则指示所述目标客户端按照预设音量播放所述待播放视频文件。
  12. 根据权利要求11所述的方法，其特征在于，所述将所述目标客户端静音，包括：
    禁止播放所述待播放视频文件。
  13. 一种视频处理方法，其特征在于，应用于电子设备，所述电子设备包括***播放模块以及FFmpeg模块，所述方法包括：
    所述***播放模块获取目标客户端发送的待播放视频文件；
    所述***播放模块判断是否支持播放所述待播放视频文件；
    若不支持，所述***播放模块将所述视频文件解析以获得所述视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；
    所述***播放模块将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块；
    所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
  14. 根据权利要求13所述的方法，其特征在于，所述***播放模块判断是否支持播放所述待播放视频文件之后，还包括：
    若不支持，所述***播放模块将所述待播放视频文件发送至FFmpeg模块；
    所述FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据。
  15. 根据权利要求14所述的方法，其特征在于，所述FFmpeg模块将所述待播放视频文件解码以获取音频播放数据和视频播放数据，包括：
    所述FFmpeg模块获取所述待播放视频文件内的音频流和视频流；
    所述FFmpeg模块根据所述视频流的类型查找到对应的视频解码器，并对所述视频流解码以获取视频播放数据，并且根据所述音频流的类型查找到对应的音频解码器并对所述音频流解码以获取音频播放数据。
  16. 根据权利要求13-15任一项所述的方法，其特征在于，所述方法还包括：
    所述FFmpeg模块将所述音频播放数据和所述视频播放数据发送至所述音视频输出模块，以指示所述音视频输出模块输出所述音频播放数据和所述视频播放数据。
  17. 一种视频处理装置，其特征在于，所述装置包括：
    获取单元，用于获取目标客户端发送的待播放视频文件；
    判断单元，用于判断是否支持播放所述待播放视频文件；
    解析单元，用于若不支持，将所述待播放视频文件解析以获得所述视频文件内的音频流和视频流；
    配置单元，用于为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识；
    处理单元，用于将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至FFmpeg模块，以使所述FFmpeg模块根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
  18. 一种视频处理装置，其特征在于，包括***播放模块以及FFmpeg模块，
    所述***播放模块用于获取目标客户端发送的待播放视频文件；判断是否支持播放所述待播放视频文件；若不支持，将所述待播放视频文件解析以获得所述视频文件内的音频流和视频流，为所述音频流配置音频解码器的标识以及为视频流配置视频解码器的标识，将所述视频流、所述音频流、视频解码器的标识以及音频解码器的标识发送至所述FFmpeg模块；
    所述FFmpeg模块用于根据所述视频解码器的标识查找到对应的视频解码器并对所述视频流解码以获取视频播放数据，并且根据所述音频解码器的标识查找到对应的音频解码器并对所述音频流解码以获取音频播放数据，其中，所述音频播放数据和视频播放数据能够被电子设备的音视频输出模块播放。
  19. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    存储器；
    ***播放模块以及FFmpeg模块；
    一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个程序配置用于执行如权利要求1-16任一项所述的方法。
  20. 一种计算机可读介质，其特征在于，所述计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行所述权利要求1-16任一项所述的方法。
PCT/CN2019/107074 2018-10-15 2019-09-20 视频处理方法、装置、电子设备及计算机可读介质 WO2020078165A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19873400.6A EP3866477A4 (en) 2018-10-15 2019-09-20 VIDEO PROCESSING PROCESS AND APPARATUS, ELECTRONIC DEVICE AND COMPUTER READABLE MEDIA
US17/231,774 US11336953B2 (en) 2018-10-15 2021-04-15 Video processing method, electronic device, and computer-readable medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811198856.9A CN109257646B (zh) 2018-10-15 2018-10-15 视频处理方法、装置、电子设备及计算机可读介质
CN201811198856.9 2018-10-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/231,774 Continuation US11336953B2 (en) 2018-10-15 2021-04-15 Video processing method, electronic device, and computer-readable medium

Publications (1)

Publication Number Publication Date
WO2020078165A1 true WO2020078165A1 (zh) 2020-04-23

Family

ID=65046263

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/107074 WO2020078165A1 (zh) 2018-10-15 2019-09-20 视频处理方法、装置、电子设备及计算机可读介质

Country Status (4)

Country Link
US (1) US11336953B2 (zh)
EP (1) EP3866477A4 (zh)
CN (1) CN109257646B (zh)
WO (1) WO2020078165A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112165649A (zh) * 2020-08-14 2021-01-01 咪咕文化科技有限公司 音视频处理方法、装置、电子设备及存储介质
CN113111200A (zh) * 2021-04-09 2021-07-13 百度在线网络技术（北京）有限公司 审核图片文件的方法、装置、电子设备和存储介质
CN113645490A (zh) * 2021-06-23 2021-11-12 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法
CN113852859A (zh) * 2021-08-19 2021-12-28 北京臻观数智科技有限公司 一种针对非标格式视频流零拷贝脱壳的方法及***
CN113905025A (zh) * 2021-09-27 2022-01-07 武汉斗鱼网络科技有限公司 一种传输流数据的方法、装置、介质及计算机设备
CN114866823A (zh) * 2022-04-02 2022-08-05 北京广播电视台 具有逻辑联动功能的超高清视频播出***及方法

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109257646B (zh) * 2018-10-15 2020-09-22 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及计算机可读介质
CN110033780B (zh) * 2019-04-07 2020-12-08 西安电子科技大学 基于FFmpeg和EMIF驱动的音视频数据传输方法
CN111356023B (zh) * 2019-12-30 2021-12-24 杭州海康威视数字技术股份有限公司 播放方式确定方法及装置
CN111309963B (zh) * 2020-01-22 2023-07-04 百度在线网络技术（北京）有限公司 音频文件处理方法、装置、电子设备及可读存储介质
WO2021218921A1 (en) * 2020-04-27 2021-11-04 Guangdong Oppo Mobile Telecommunications Corp., Ltd. System and method for extraction of a video thumbnail from a video file
CN111641867B (zh) * 2020-04-29 2023-09-26 中国平安财产保险股份有限公司 视频输出方法、装置、电子设备及存储介质
CN112019917A (zh) * 2020-07-28 2020-12-01 厦门快商通科技股份有限公司 音频数据提取方法、装置、设备及存储介质
CN112511840B (zh) * 2020-12-24 2023-02-17 北京睿芯高通量科技有限公司 一种基于ffmpeg与硬件加速设备的解码***及方法
CN113395585B (zh) * 2021-02-09 2024-05-31 腾讯科技（北京）有限公司 视频检测方法、视频播放控制方法、装置和电子设备
CN113727110A (zh) * 2021-08-27 2021-11-30 猪八戒股份有限公司 将h264格式转换为mp4格式的方法、装置及介质
CN115529491B (zh) * 2022-01-10 2023-06-06 荣耀终端有限公司 一种音视频解码的方法、音视频解码的装置以及终端设备
CN114449295A (zh) * 2022-01-30 2022-05-06 京东方科技集团股份有限公司 视频处理方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104980788A (zh) * 2015-02-11 2015-10-14 腾讯科技（深圳）有限公司 视频解码方法及装置
US20160029002A1 (en) * 2014-07-26 2016-01-28 Soeren Balko Platform-agnostic Video Player For Mobile Computing Devices And Desktop Computers
CN108235096A (zh) * 2018-01-18 2018-06-29 湖南快乐阳光互动娱乐传媒有限公司 移动终端硬解码智能切换软解码播放视频的方法
CN108600813A (zh) * 2018-05-17 2018-09-28 上海七牛信息技术有限公司 一种媒体文件播放方法、装置以及媒体播放***
CN109257646A (zh) * 2018-10-15 2019-01-22 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及计算机可读介质

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2188808B1 (en) * 2007-08-17 2016-07-06 Linius (AUST) Pty Ltd Method and system for content delivery
US8903897B2 (en) * 2011-07-08 2014-12-02 Google Inc. System and method for providing interactive content to non-native application environments
CN102685507B (zh) * 2011-07-12 2014-07-09 天脉聚源(北京)传媒科技有限公司 一种mpeg-2传输流解码音视频同步方法和***
CN102567468B (zh) * 2011-12-06 2014-06-04 上海聚力传媒技术有限公司 一种用于对媒体文件的播放音量进行调节的方法与设备
CA2791935A1 (en) * 2012-03-30 2013-09-30 Disternet Technology, Inc. Transcoding system and method
KR101920298B1 (ko) * 2012-07-17 2018-11-20 현대모비스 주식회사 Ｄａｂ 방송 수신 장치 및 그 ｄｍｂ 방송 수신 방법
US9449645B2 (en) * 2013-07-03 2016-09-20 Salutations 365 LLC Video compilation greeting system and method
CN103731672A (zh) * 2013-12-16 2014-04-16 乐视致新电子科技（天津）有限公司 一种音视频解码方法及智能电视
CN104394456B (zh) * 2014-11-20 2017-12-01 福建星网视易信息***有限公司 一种嵌入式环境下音视频解码的方法及装置
CN106817354A (zh) * 2015-12-01 2017-06-09 阿里巴巴集团控股有限公司 一种视频流传输方法、设备和***
CN105898538A (zh) * 2015-12-14 2016-08-24 乐视网信息技术（北京）股份有限公司 用于安卓平台的播放方法、装置及移动终端设备
CN106060627A (zh) * 2016-06-15 2016-10-26 乐视控股（北京）有限公司 一种基于多路直播的音频处理方法及装置
CN108010534A (zh) * 2016-11-01 2018-05-08 北京酷我科技有限公司 一种音频文件播放方法及装置
KR102130429B1 (ko) * 2016-11-07 2020-07-07 한화테크윈 주식회사 멀티미디어 수신 장치에서 디코딩을 수행하는 방법 및 멀티미디어 장치
US20190069006A1 (en) * 2017-08-29 2019-02-28 Western Digital Technologies, Inc. Seeking in live-transcoded videos
CN107995522A (zh) 2017-12-19 2018-05-04 深圳市沃特沃德股份有限公司 车载***播放视频的方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160029002A1 (en) * 2014-07-26 2016-01-28 Soeren Balko Platform-agnostic Video Player For Mobile Computing Devices And Desktop Computers
CN104980788A (zh) * 2015-02-11 2015-10-14 腾讯科技（深圳）有限公司 视频解码方法及装置
CN108235096A (zh) * 2018-01-18 2018-06-29 湖南快乐阳光互动娱乐传媒有限公司 移动终端硬解码智能切换软解码播放视频的方法
CN108600813A (zh) * 2018-05-17 2018-09-28 上海七牛信息技术有限公司 一种媒体文件播放方法、装置以及媒体播放***
CN109257646A (zh) * 2018-10-15 2019-01-22 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及计算机可读介质

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112165649A (zh) * 2020-08-14 2021-01-01 咪咕文化科技有限公司 音视频处理方法、装置、电子设备及存储介质
CN113111200A (zh) * 2021-04-09 2021-07-13 百度在线网络技术（北京）有限公司 审核图片文件的方法、装置、电子设备和存储介质
CN113111200B (zh) * 2021-04-09 2024-05-24 百度在线网络技术（北京）有限公司 审核图片文件的方法、装置、电子设备和存储介质
CN113645490A (zh) * 2021-06-23 2021-11-12 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法
CN113645490B (zh) * 2021-06-23 2023-05-09 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法
CN113852859A (zh) * 2021-08-19 2021-12-28 北京臻观数智科技有限公司 一种针对非标格式视频流零拷贝脱壳的方法及***
CN113905025A (zh) * 2021-09-27 2022-01-07 武汉斗鱼网络科技有限公司 一种传输流数据的方法、装置、介质及计算机设备
CN113905025B (zh) * 2021-09-27 2023-06-16 武汉斗鱼网络科技有限公司 一种传输流数据的方法、装置、介质及计算机设备
CN114866823A (zh) * 2022-04-02 2022-08-05 北京广播电视台 具有逻辑联动功能的超高清视频播出***及方法

Also Published As

Publication number Publication date
CN109257646A (zh) 2019-01-22
US20210235153A1 (en) 2021-07-29
US11336953B2 (en) 2022-05-17
EP3866477A1 (en) 2021-08-18
CN109257646B (zh) 2020-09-22
EP3866477A4 (en) 2021-10-20

Similar Documents

Publication Publication Date Title
WO2020078165A1 (zh) 视频处理方法、装置、电子设备及计算机可读介质
CN110740363B (zh) 投屏方法和***、电子设备
US20170111414A1 (en) Video playing method and device
US11490173B2 (en) Switch of audio and video
TWI502977B (zh) 影音播放裝置、影音處理裝置、系統以及方法
US11792464B2 (en) Determining context to initiate interactivity
CN109587546B (zh) 视频处理方法、装置、电子设备和计算机可读介质
CN103024603B (zh) 一种用于解决播放网络视频时短时停顿的装置及方法
CN108932948B (zh) 音频数据处理方法、装置、计算机设备和计算机可读存储介质
CN102074257A (zh) 软硬件结合解码的通用多媒体播放设备及其播放方法
CN110996160B (zh) 视频处理方法、装置、电子设备及计算机可读取存储介质
WO2018130173A1 (zh) 配音方法、终端设备、服务器及存储介质
CN111510755A (zh) 音视频的切换方法、装置、计算机设备及可读存储介质
CN111050201A (zh) 数据处理方法、装置、电子设备及存储介质
WO2018192183A1 (zh) 无线传屏中视频文件的处理方法及装置
US10819951B2 (en) Recording video from a bitstream
CN114666516A (zh) 显示设备及流媒体文件合成方法
CN111352605A (zh) 一种音频播放、发送的方法及装置
US11985290B2 (en) Information processing device, information processing method, reproduction processing device, and reproduction processing method
KR20140117889A (ko) 클라이언트 장치, 서버 장치, 멀티미디어 리디렉션 시스템 및 그 방법
WO2020107168A1 (zh) 视频解码方法、装置、电子设备、计算机可读存储介质
KR20170010574A (ko) 정보처리장치, 영상처리장치 및 그들의 제어방법
CN115604238A (zh) 物联网操作***的音视频处理方法、装置、设备和介质
CN114422840A (zh) 音轨切换的方法及装置
CN113261300B (zh) 音频发送、播放的方法及智能电视

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19873400

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019873400

Country of ref document: EP

Effective date: 20210510