CN114071149A - Distributed video transcoding method and related device - Google Patents

Distributed video transcoding method and related device Download PDF

Info

Publication number
CN114071149A
CN114071149A CN202010749811.7A CN202010749811A CN114071149A CN 114071149 A CN114071149 A CN 114071149A CN 202010749811 A CN202010749811 A CN 202010749811A CN 114071149 A CN114071149 A CN 114071149A
Authority
CN
China
Prior art keywords
transcoding
video
file
uploading
task
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010749811.7A
Other languages
Chinese (zh)
Other versions
CN114071149B (en
Inventor
张平
吕宇
皮志超
吴可夫
谭斌
周培慧
谢喆
黄成满
贺谦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuzhou CRRC Times Electric Co Ltd
Original Assignee
Zhuzhou CRRC Times Electric Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuzhou CRRC Times Electric Co Ltd filed Critical Zhuzhou CRRC Times Electric Co Ltd
Priority to CN202010749811.7A priority Critical patent/CN114071149B/en
Publication of CN114071149A publication Critical patent/CN114071149A/en
Application granted granted Critical
Publication of CN114071149B publication Critical patent/CN114071149B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/127—Prioritisation of hardware or computational resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention provides a distributed video transcoding method and a related device, wherein the method comprises the steps of controlling a video uploading terminal through a transcoding control server, realizing a network-based distributed transcoding system, distributing transcoding tasks by using a network, realizing controlled distributed transcoding at the video uploading terminal, managing and scheduling the transcoding tasks of all the video uploading terminals by the transcoding control server, realizing the reutilization and balance of transcoding calculation performance under the condition of unbalanced hardware configuration, and achieving the purpose of efficient and balanced large-scale video transcoding. And the occupation conditions of a CPU, a hard disk and contents are monitored, and high-priority tasks such as user operation are protected. Repeated transcoding is avoided when a transcoding task is recovered through automatic rescanning and repeated content elimination; when the network is interrupted, an independent transcoding task execution mode is adopted, so that the transcoding process is not interrupted, and uploading is automatically recovered when the network is recovered. Better system adaptability and stability are achieved through exception handling.

Description

Distributed video transcoding method and related device
Technical Field
The present invention relates to the technical field of video data encoding and decoding, and more particularly, to a distributed video transcoding method and related apparatus.
Background
The video transcoding is to convert a video code stream which has been compressed and encoded into another video code stream so as to adapt to different network bandwidths, different terminal processing capabilities and different user requirements. Video transcoding is essentially a process of decoding first and then encoding, so that video code streams before and after conversion can conform to the same video encoding standard and can also conform to different video encoding standards.
The video transcoding task usually consumes a lot of system computing resources, including CPU processing power and hard disk I/O bandwidth occupation, so that it is not efficient to perform large-scale video transcoding by a single terminal. Therefore, there is a need for a method of transcoding video with high efficiency.
Disclosure of Invention
In view of this, the present invention provides a distributed video transcoding method and a related apparatus, which are intended to improve the efficiency of video transcoding.
In order to achieve the above object, the following solutions are proposed:
in a first aspect, a distributed video transcoding method is provided, which is applied to a video uploading terminal, and the distributed video transcoding method includes:
scanning a local video file to be transcoded, and establishing a transcoding task file, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding duration of the video file to be transcoded;
uploading the transcoding task file to a transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
receiving the transcoding plan file issued by the transcoding control server, and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to a video data server.
Optionally, in the process of performing transcoding according to the updated transcoding task file, if there are at least two video files to be transcoded, in the process of uploading one transcoded video file to the video data server, a next video file to be transcoded is transcoded at the same time.
Optionally, in the process of uploading the transcoded video file to the video data server, the method further includes:
the network connection speed of the video data server and the transcoding control server is detected in real time, and the network connection speed is periodically sent to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of the corresponding video uploading terminal.
Optionally, in the process of performing transcoding, the method further includes:
and detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the device in real time, when any one of the occupancy rates is larger than the corresponding occupancy rate threshold value, pausing transcoding after the video file currently being transcoded is finished, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
Optionally, in the process of performing transcoding, the method further includes:
when a transcoding task is lost, scanning a local video file to be transcoded, and reestablishing a transcoding task file, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file;
and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reestablished transcoding task file.
Optionally, when the network is interrupted, the uploading of the transcoded video file to the video data server is suspended, and the transcoding process is continuously executed.
In a second aspect, a distributed video transcoding method is provided, and is applied to a transcoding control server, where the distributed video transcoding method includes:
receiving transcoding task files uploaded by each video uploading terminal, wherein the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by a transcoding control server;
and issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update the transcoding task files of the video uploading terminals according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
Optionally, the video uploading terminal stored in the transcoding control server specifically includes:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
Optionally, after the transcoding plan file is issued to all stored video uploading terminals, the method further includes:
receiving a network connection rate of a video uploading terminal, wherein the network connection rate is a real-time network connection rate of the video uploading terminal and a video data server;
and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
Optionally, the transcoding control server further stores a performance index of the video uploading terminal, where the performance index is calculated from the cpu master frequency, the memory capacity, the hard disk capacity, and the network width of the video uploading terminal, and the generating of the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically includes:
for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file;
for each video uploading terminal, calculating the ratio of the estimated total transcoding time to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
In a third aspect, a distributed video transcoding device is provided, which is applied to a video uploading terminal, and includes:
the transcoding task establishing unit is used for scanning a local video file to be transcoded and establishing a transcoding task file, wherein the transcoding task file comprises the file name, the file path, the file size, the file format and the estimated transcoding time length of the video file to be transcoded;
the transcoding task uploading unit is used for uploading the transcoding task file to the transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
the transcoding task updating unit is used for receiving the transcoding plan file issued by the transcoding control server and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and the transcoding uploading unit is used for executing a transcoding process according to the updated transcoding task file and uploading the transcoded video file to the video data server.
Optionally, the transcoding uploading unit is specifically configured to:
and in the process of executing transcoding according to the updated transcoding task file, if at least two video files to be transcoded exist, transcoding a next video file to be transcoded in the process of uploading one transcoded video file to a video data server.
Optionally, the transcoding uploading unit is further configured to:
in the process of uploading the transcoded video file to a video data server, detecting the network connection rate of a video uploading unit and the video data server in real time, and periodically sending the network connection rate to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of a corresponding video uploading terminal.
Optionally, the distributed video transcoding apparatus further includes:
and the pause continuous-transcoding unit is used for detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the video uploading terminal in real time in the transcoding execution process, pausing transcoding after finishing the video file currently being transcoded when any one of the occupancy rates is greater than the corresponding occupancy rate threshold value, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
Optionally, the distributed video transcoding apparatus further includes:
the first exception handling unit is used for scanning a local video file to be transcoded and reestablishing a transcoding task file if a transcoding task is lost in the transcoding executing process, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file; and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reconstructed transcoding task file.
Optionally, the distributed video transcoding apparatus further includes:
and the second exception handling unit is used for suspending uploading of the transcoded video file to the video data server and continuing to execute the transcoding process when the network is interrupted.
In a fourth aspect, a distributed video transcoding apparatus is provided, which is applied to a transcoding control server, and includes:
the transcoding task receiving unit is used for receiving transcoding task files uploaded by all the video uploading terminals, and the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
the transcoding distribution unit is used for generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
and the transcoding plan issuing unit is used for issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update own transcoding task files according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
Optionally, the video uploading terminal stored in the transcoding control server specifically includes:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
Optionally, the distributed video transcoding apparatus further includes:
the video uploading system comprises an uploading time dynamic estimation unit, a video data server and a video uploading terminal, wherein the uploading time dynamic estimation unit is used for receiving the network connection rate of the video uploading terminal, and the network connection rate is the real-time network connection rate of the video uploading terminal and the video data server; and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
Optionally, the transcoding control server further stores a performance index of the video uploading terminal, where the performance index is calculated from the cpu master frequency, the memory capacity, the hard disk capacity, and the network width of the video uploading terminal, and the transcoding allocating unit specifically includes:
the pre-estimated total transcoding time length unit is used for calculating and obtaining the pre-estimated total transcoding time length for transcoding the local video file to be transcoded of each video uploading terminal according to the uploaded transcoding task file;
the task load calculation unit is used for calculating the ratio of the pre-estimated total transcoding time length to the performance index for each video uploading terminal, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and the task balancing unit is used for distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
In a fifth aspect, a distributed video transcoding system is provided, which comprises a transcoding control server and at least two video uploading terminals;
the video uploading terminal is used for scanning a local video file to be transcoded, establishing a transcoding task file and uploading the transcoding task file to the transcoding control server, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding time of the video file to be transcoded;
the transcoding control server is used for generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal and sending the transcoding plan file to all the stored video uploading terminals, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
and the video uploading terminal is also used for updating the transcoding task file according to the transcoding plan corresponding to the video uploading terminal in the transcoding plan file, executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to the video data server.
Optionally, the video uploading terminal stored in the transcoding control server specifically includes:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
Optionally, the video uploading terminal is further configured to, in the process of performing transcoding according to the updated transcoding task file, if there are at least two video files to be transcoded, transcode a next video file to be transcoded while uploading a transcoded video file to the video data server.
Optionally, the video uploading terminal is further configured to detect a network connection rate between the video uploading terminal and the video data server in real time during the process of uploading the transcoded video file to the video data server, and periodically send the network connection rate to the transcoding control server;
and the transcoding control server is also used for dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
Optionally, the video uploading terminal is further configured to detect a CPU occupancy rate, a memory occupancy rate, and a hard disk capacity occupancy rate of the video uploading terminal in real time during transcoding execution, and when any one of the occupancy rates is greater than a corresponding occupancy rate threshold, pause transcoding after a currently transcoded video file is completed, and continue a subsequent transcoding task after the corresponding occupancy rate meets a preset condition.
Optionally, the video uploading terminal is further configured to scan a local video file to be transcoded and reestablish a transcoding task file if a transcoding task is detected to be lost in the transcoding process, or acquire a transcoding plan file from the transcoding control server and reestablish the transcoding task file; and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reconstructed transcoding task file.
Optionally, the video uploading terminal is further configured to suspend uploading of the transcoded video file to the video data server when the network is interrupted, and continue to execute the transcoding process.
Optionally, the transcoding control server is further configured to store a performance index of the video uploading terminal, where the performance index is obtained by calculating a cpu dominant frequency, a memory capacity, a hard disk capacity, and a network width of the video uploading terminal;
the process that the transcoding control server generates the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically comprises the following steps:
for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file;
for each video uploading terminal, calculating the ratio of the estimated total transcoding time to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
A sixth aspect provides a readable storage medium, on which a program is stored, which when executed by a processor, implements the steps of the distributed video transcoding method as in any one of the first aspect.
In a seventh aspect, a readable storage medium is provided, on which a program is stored, which when executed by a processor, implements the steps of the distributed video transcoding method according to any one of the second aspect.
In an eighth aspect, an electronic device is provided, comprising a memory for storing a program and a processor; the processor is configured to execute the program to implement the steps of any one of the distributed video transcoding methods in the first aspect.
In a ninth aspect, an electronic device is provided, comprising a memory for storing a program and a processor; the processor is configured to execute the program to implement the steps of any one of the distributed video transcoding methods in the second aspect
Compared with the prior art, the technical scheme of the invention has the following advantages:
the video uploading terminal is controlled through the transcoding control server, a network-based distributed transcoding system is realized, a network is used for distributing transcoding tasks, controlled distributed transcoding is realized on the video uploading terminal, the transcoding control server manages and schedules the transcoding tasks of all the video uploading terminals, the reutilization and balance of transcoding calculation performance are realized under the condition of unbalanced hardware configuration, and the purpose of efficient and balanced large-scale video transcoding is achieved.
Furthermore, the occupation conditions of the CPU, the hard disk and the content are monitored in real time so as to protect high-priority tasks such as user operation, data copying and the like.
Further, repeated transcoding during transcoding task recovery is avoided through automatic rescanning and repeated content elimination; and when the network is interrupted, an independent transcoding task execution mode is adopted, so that the transcoding process is not interrupted, and uploading is automatically recovered when the network is recovered. Better system adaptability and stability are achieved through exception handling.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic diagram of a distributed video transcoding system according to an embodiment of the present invention;
fig. 2 is a flowchart of a distributed video transcoding method applied to a video uploading terminal according to an embodiment of the present invention;
fig. 3 is a flowchart of a distributed video transcoding method applied to a transcoding control server according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a distributed video transcoding apparatus applied to a video uploading terminal according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a distributed video transcoding apparatus applied to a transcoding control server according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a distributed video transcoding system provided for an embodiment of the present invention includes a transcoding control server and at least two video uploading terminals.
And the video uploading terminal is used for scanning the local video file to be transcoded, establishing a transcoding task file and uploading the transcoding task file to the transcoding control server. The transcoding task file comprises the file name, the file path, the file size, the file format and the pre-estimated transcoding time of the video file to be transcoded. The estimated transcoding duration can be calculated through the file size, the SCR time length of the video file and the transcoding multiplying power. The transcoding task file may specifically be an XML format file. And the video uploading terminal is a terminal registered in the transcoding controller in advance. And allowing the transcoding control server to perform transcoding control scheduling by using the registration terminal, namely the video uploading terminal in the invention. The terminal registration specifically refers to storing the unique number of the terminal in a transcoding control server, and then performing transcoding control scheduling according to the number.
And the transcoding control server is used for generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal and sending the transcoding plan file to all the stored video uploading terminals, wherein the transcoding plan file comprises the transcoding plans of all the video uploading terminals stored by the transcoding control server. And the transcoding control server determines the transcoding task amount required to be completed by each video uploading terminal according to the principle of optimal performance priority and shortest parallel total transcoding time length, so that a transcoding plan file is formed.
The video uploading terminal is further used for updating the transcoding task file according to the transcoding plan corresponding to the video uploading terminal in the transcoding plan file, executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to the video data server. It should be noted that, if the video uploading terminal receives the transcoding plan file in the transcoding process, the transcoding task file is updated synchronously in the current transcoding process, so that the execution of the transcoding process of the video uploading terminal is not affected, that is, the current transcoding task is not interrupted.
In some embodiments, the video uploading terminal stored in the transcoding control server is specifically a terminal in which the CPU dominant frequency is greater than a preset dominant frequency threshold, the memory is greater than a preset memory threshold, the hard disk capacity is greater than a preset hard disk capacity threshold, and the network broadband is greater than a preset network broadband threshold. For example, the terminal registers to a transcoding control server in the network first, and can report the CPU main frequency, memory capacity, hard disk capacity, network bandwidth and network card MAC address of the terminal in the form of an XML registration file; the transcoding control server determines whether the terminal is allowed to register according to the limited basic configuration requirements (such as CPU main frequency at least 2GHz, memory at least 2GB, hard disk capacity at least 128GB, network bandwidth at least 100Mb/s), and issues a unique number in the network to the registered terminal, wherein the number is generated through a network card MAC address.
In some embodiments, the video uploading terminal is further configured to, in the process of performing transcoding according to the updated transcoding task file, if there are at least two video files to be transcoded, transcode a next video file to be transcoded while uploading a transcoded video file to the video data server, so that the video uploading and the video transcoding are performed in parallel.
In some specific embodiments, the video uploading terminal is further configured to detect a network connection rate between the video uploading terminal and the video data server in real time during the process of uploading the transcoded video file to the video data server, and periodically send the network connection rate to the transcoding control server; and the transcoding control server is also used for dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate. When the video uploading terminal detects that the network connection rate is lower than a preset network connection rate threshold value, the video uploading terminal outputs prompt information of abnormal network connection rate to prompt a user to check the network connection state. After judging that the network connection rate of a certain video uploading terminal is lower than the preset network connection rate threshold, the transcoding control server appropriately reduces the traffic processing amount when adjusting the transcoding plan next time, for example, the task amount may be multiplied by 0.8 to reduce the task amount.
In some specific embodiments, the video uploading terminal is further configured to detect a CPU occupancy rate, a memory occupancy rate, and a hard disk capacity occupancy rate of the video uploading terminal in real time during transcoding execution, and when any one of the occupancy rates is greater than a corresponding occupancy rate threshold, pause transcoding after a currently transcoded video file is completed, and continue a subsequent transcoding task after the corresponding occupancy rate meets a preset condition. Namely, the transcoding process is executed according to the lower priority, so that high-priority tasks such as user operation and data copying are protected, and the influence on operators of the video uploading terminal is reduced.
In some specific embodiments, the video uploading terminal is further configured to scan a local video file to be transcoded and reestablish a transcoding task file if a transcoding task is detected to be lost in the transcoding process, or acquire a transcoding plan file from the transcoding control server and reestablish the transcoding task file; and according to the uploaded video file list, removing the uploaded video files from the re-established transcoding task files, and avoiding repeated transcoding after the transcoding task files are recovered.
In some embodiments, the video uploading terminal is further configured to suspend uploading of the transcoded video file to the video data server and continue to perform the transcoding process when the network is interrupted. After the network is recovered, the transcoding control server automatically registers and reports transcoding state information, and simultaneously, all transcoded video files are transmitted to the video data server.
In some specific embodiments, the transcoding control server is further configured to store the performance index of the video uploading terminal. The performance index is obtained by calculating the CUP dominant frequency, the memory capacity, the hard disk capacity and the network width of the video uploading terminal. When the terminal registers, the transcoding control service performs weight distribution on computer parameters (such as CUP main frequency, memory capacity, hard disk capacity and network broadband) with large transcoding influence degree, divides each parameter index into multiple grades, performs automatic scoring, and obtains the total score by accumulating the sum of each parameter, and normalizes the total score to 0-100 sections to obtain the performance index of the video uploading terminal,
the process that the transcoding control server generates the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically comprises the following steps: for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file; for each video uploading terminal, calculating the ratio of the estimated total transcoding time length to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal; and for the video uploading terminal with the task load larger than the preset load threshold, distributing part of transcoding tasks of the video uploading terminal to the video uploading terminal with the minimum task load. For example, when the task load of the video uploading terminal A is too large, the video uploading terminal is considered to be incapable of efficiently completing a transcoding task, the transcoding control server issues an instruction to the video uploading terminal, and uploading of a transcoded file is suspended; simultaneously, the transcoding control server provides an address of a video uploading terminal B with the minimum task load, the video uploading terminal A sends 1/3 video files to be transcoded to the video uploading terminal B in an FTP mode, the transcoding task files are updated simultaneously, and A, B two video uploading terminals immediately send the updated transcoding task files to the transcoding control server; after the transcoding video file is transferred, the transcoded video file of the video uploading terminal A is uploaded and recovered, and the transcoding task is continuously carried out in the process and is not influenced. And the estimated total transcoding time for the video uploading terminal to transcode the local video file to be transcoded is the accumulated sum of the estimated transcoding time of each video file to be transcoded in the transcoding task file.
In order to realize the state monitoring of each video uploading terminal by the transcoding control server, the video uploading terminal can be set to periodically send monitoring information of the transcoding and uploading states to the transcoding control server by a UDP protocol. The monitoring information of the transcoding and uploading state comprises the number of transcoded files, the number of residual transcoded files, the number of transcoding failure files, the number of uploaded files and the network connection rate of the video uploading terminal and the video data server. The transcoding control server calculates the completion rate of transcoding tasks through the transcoding plan file and the state monitoring information of the transcoding and uploading states of the video uploading terminals, and predicts the total time for each video uploading terminal to complete transcoding and uploading;
and the transcoding task completion rate is (the number of transcoded files/the total number of files to be transcoded in the transcoding task file of the current video uploading terminal) 100%. And the total time for completing transcoding and uploading is the larger value of the estimated total transcoding time for the video uploading terminal to transcode the local video file to be transcoded and the estimated total time for completing uploading the current file. The estimated total time for completing uploading the current file is obtained by dividing the total size of the file to be uploaded by the average uploading rate, the average uploading rate is the average value of the network connection rate detected before, and the initial value is the network broadband value during registration. Through the state monitoring of each video uploading terminal, whether the existing equipment can meet the requirement of data volume or not is known. If the equipment is insufficient, the terminals can be increased in time or the terminals with stronger service performance can be used.
Referring to fig. 2, a distributed video transcoding method applied to a video uploading terminal is provided, where the method includes the following steps:
s21: and scanning the local video file to be transcoded, and establishing a transcoding task file.
The transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding time of the video file to be transcoded.
S22: and uploading the transcoding task file to a transcoding control server, so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal.
The transcoding plan file contains the transcoding plans of all video uploading terminals stored by the transcoding control server.
S23: and receiving a transcoding plan file issued by the transcoding control server, and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file.
S24: and executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to a video data server.
In some embodiments, in the process of performing transcoding according to the updated transcoding task file, if there are at least two video files to be transcoded, a next video file to be transcoded is transcoded while uploading a transcoded video file to the video data server.
In some specific embodiments, in the process of uploading the transcoded video file to the video data server, the method further includes:
the network connection speed of the video data server and the transcoding control server is detected in real time, and the network connection speed is periodically sent to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of the corresponding video uploading terminal.
In some specific embodiments, in the performing of the transcoding, the method further includes:
and detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the device in real time, when any one of the occupancy rates is larger than the corresponding occupancy rate threshold value, pausing transcoding after the video file currently being transcoded is finished, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
In some specific embodiments, in the performing of the transcoding, the method further includes:
when a transcoding task is lost, scanning a local video file to be transcoded, and reestablishing a transcoding task file, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file;
and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reestablished transcoding task file.
In some embodiments, when the network is interrupted, the uploading of the transcoded video file to the video data server is suspended, and the transcoding process is continued.
Referring to fig. 3, a distributed video transcoding method applied to a transcoding control server is provided for an embodiment of the present invention, where the method includes:
s31: and receiving the transcoding task file uploaded by each video uploading terminal.
The transcoding task file comprises the file name, the file path, the file size, the file format and the pre-estimated transcoding time of the video file to be transcoded.
S32: and generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises the transcoding plans of all the video uploading terminals stored by the transcoding control server.
S33: and issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update the transcoding task files of the video uploading terminals according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
In some specific embodiments, the video uploading terminal stored by the transcoding control server specifically includes: the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
In some specific embodiments, after issuing the transcoding plan file to all stored video uploading terminals, the method further includes:
receiving a network connection rate of a video uploading terminal, wherein the network connection rate is a real-time network connection rate of the video uploading terminal and a video data server;
and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
In some specific embodiments, the transcoding control server further stores a performance index of the video uploading terminal, where the performance index is calculated from a cpu dominant frequency, a memory capacity, a hard disk capacity, and a network width of the video uploading terminal, and the generating of the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically includes:
for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file;
for each video uploading terminal, calculating the ratio of the estimated total transcoding time to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
Referring to fig. 4, a distributed video transcoding apparatus applied to a video uploading terminal is provided for an embodiment of the present invention, and the apparatus includes: a transcoding task establishing unit 41, a transcoding task uploading unit 42, a transcoding task updating unit 43, and a transcoding uploading unit 44.
The transcoding task establishing unit 41 is configured to scan a local video file to be transcoded, and establish a transcoding task file, where the transcoding task file includes a file name, a file path, a file size, a file format, and an estimated transcoding duration of the video file to be transcoded;
the transcoding task uploading unit 42 is configured to upload the transcoding task file to the transcoding control server, so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, where the transcoding plan file includes transcoding plans of all the video uploading terminals stored by the transcoding control server;
a transcoding task updating unit 43, configured to receive the transcoding plan file issued by the transcoding control server, and update the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and the transcoding uploading unit 44 is configured to execute a transcoding process according to the updated transcoding task file, and upload the transcoded video file to the video data server.
In some specific embodiments, the transcoding uploading unit is specifically configured to:
and in the process of executing transcoding according to the updated transcoding task file, if at least two video files to be transcoded exist, transcoding a next video file to be transcoded in the process of uploading one transcoded video file to a video data server.
In some specific embodiments, the transcoding uploading unit is further configured to:
in the process of uploading the transcoded video file to a video data server, detecting the network connection rate of a video uploading unit and the video data server in real time, and periodically sending the network connection rate to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of a corresponding video uploading terminal.
In some specific embodiments, the distributed video transcoding apparatus further includes:
and the pause continuous-transcoding unit is used for detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the video uploading terminal in real time in the transcoding execution process, pausing transcoding after finishing the video file currently being transcoded when any one of the occupancy rates is greater than the corresponding occupancy rate threshold value, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
In some specific embodiments, the distributed video transcoding apparatus further includes:
the first exception handling unit is used for scanning a local video file to be transcoded and reestablishing a transcoding task file if a transcoding task is lost in the transcoding executing process, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file; and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reconstructed transcoding task file.
In some specific embodiments, the distributed video transcoding apparatus further includes:
and the second exception handling unit is used for suspending uploading of the transcoded video file to the video data server and continuing to execute the transcoding process when the network is interrupted.
Referring to fig. 5, a distributed video transcoding apparatus applied to a transcoding control server is provided for an embodiment of the present invention, and the apparatus includes: a transcoding task receiving unit 51, a transcoding distribution unit 52, and a transcoding plan issuing unit 53.
The transcoding task receiving unit 51 is configured to receive a transcoding task file uploaded by each video uploading terminal, where the transcoding task file includes a file name, a file path, a file size, a file format, and a pre-estimated transcoding duration of the video file to be transcoded;
the transcoding distribution unit 52 is configured to generate a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, where the transcoding plan file includes transcoding plans of all the video uploading terminals stored by the transcoding control server;
and the transcoding plan issuing unit 53 is configured to issue the transcoding plan file to all stored video uploading terminals, so that the video uploading terminals update their own transcoding task files according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
In a specific embodiment, the video uploading terminal stored in the transcoding control server specifically includes:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
In a specific embodiment, the distributed video transcoding apparatus further includes:
the video uploading system comprises an uploading time dynamic estimation unit, a video data server and a video uploading terminal, wherein the uploading time dynamic estimation unit is used for receiving the network connection rate of the video uploading terminal, and the network connection rate is the real-time network connection rate of the video uploading terminal and the video data server; and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
In a specific embodiment, the transcoding control server further stores a performance index of the video uploading terminal, where the performance index is calculated from a cpu dominant frequency, a memory capacity, a hard disk capacity, and a network width of the video uploading terminal, and the transcoding allocating unit specifically includes:
the pre-estimated total transcoding time length unit is used for calculating and obtaining the pre-estimated total transcoding time length for transcoding the local video file to be transcoded of each video uploading terminal according to the uploaded transcoding task file;
the task load calculation unit is used for calculating the ratio of the pre-estimated total transcoding time length to the performance index for each video uploading terminal, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and the task balancing unit is used for distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
Referring to fig. 6, an embodiment of the invention provides an electronic device. The hardware structure of the electronic device may include: at least one processor 61, at least one communication interface 62, at least one memory 63 and at least one communication bus 64; and the processor 61, the communication interface 62 and the memory 63 are communicated with each other through a communication bus 64.
Processor 61 may be, in some embodiments, a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), one or more Integrated circuits configured to implement embodiments of the present invention, or the like.
The communication interface 62 may include a standard wired interface, a wireless interface (e.g., WI-FI interface). Are commonly used to establish communication connections with other electronic devices or systems.
The memory 63 includes at least one type of readable storage medium. The readable storage medium may be an NVM (non-volatile memory) such as flash memory, hard disk, multimedia card, card-type memory, etc. The readable storage medium may also be a high-speed RAM (random access memory) memory. The readable storage medium may in some embodiments be an internal storage unit of the electronic device, for example a hard disk of the electronic device. In other embodiments, the readable storage medium may be an external storage device of the electronic device, such as a plug-in hard disk, an SMC (Smart Media Card), an SD (Secure Digital) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device.
Wherein the memory 63 stores a computer program, and the processor 61 may call the computer program stored in the memory 63, the computer program being configured to:
scanning a local video file to be transcoded, and establishing a transcoding task file, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding duration of the video file to be transcoded;
uploading the transcoding task file to a transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
receiving the transcoding plan file issued by the transcoding control server, and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to a video data server.
The computer program may also be for:
receiving transcoding task files uploaded by each video uploading terminal, wherein the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by a transcoding control server;
and issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update the transcoding task files of the video uploading terminals according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
The refinement function and the extension function of the program may be referred to as described above.
FIG. 6 shows only an electronic device having components 61-64, but it is understood that not all of the shown components are required to be implemented, and that more or fewer components can be implemented instead.
Embodiments of the present invention also provide a readable storage medium storing a program adapted to be executed by a processor, the program being configured to:
scanning a local video file to be transcoded, and establishing a transcoding task file, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding duration of the video file to be transcoded;
uploading the transcoding task file to a transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
receiving the transcoding plan file issued by the transcoding control server, and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to a video data server.
The computer program may also be for:
receiving transcoding task files uploaded by each video uploading terminal, wherein the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by a transcoding control server;
and issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update the transcoding task files of the video uploading terminals according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
The refinement function and the extension function of the program may be referred to as described above.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are mainly described as different from other embodiments, the same and similar parts in the embodiments may be referred to each other, and the features described in the embodiments in the present description may be replaced with each other or combined with each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (32)

1. A distributed video transcoding method is applied to a video uploading terminal, and comprises the following steps:
scanning a local video file to be transcoded, and establishing a transcoding task file, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding duration of the video file to be transcoded;
uploading the transcoding task file to a transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
receiving the transcoding plan file issued by the transcoding control server, and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to a video data server.
2. The distributed video transcoding method of claim 1, wherein in the transcoding process performed according to the updated transcoding task file, if there are at least two video files to be transcoded, a next video file to be transcoded is transcoded while one transcoded video file is uploaded to a video data server.
3. The distributed video transcoding method of claim 1, wherein in the process of uploading the transcoded video file to the video data server, the method further comprises:
the network connection speed of the video data server and the transcoding control server is detected in real time, and the network connection speed is periodically sent to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of the corresponding video uploading terminal.
4. The distributed video transcoding method of claim 1, wherein in performing transcoding, further comprising:
and detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the device in real time, when any one of the occupancy rates is larger than the corresponding occupancy rate threshold value, pausing transcoding after the video file currently being transcoded is finished, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
5. The distributed video transcoding method of claim 1, wherein in performing transcoding, further comprising:
when a transcoding task is lost, scanning a local video file to be transcoded, and reestablishing a transcoding task file, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file;
and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reestablished transcoding task file.
6. The distributed video transcoding method of claim 1, wherein when the network is interrupted, uploading of the transcoded video file to the video data server is suspended, and the transcoding process is continued.
7. A distributed video transcoding method is applied to a transcoding control server, and comprises the following steps:
receiving transcoding task files uploaded by each video uploading terminal, wherein the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, wherein the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by a transcoding control server;
and issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update the transcoding task files of the video uploading terminals according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
8. The distributed video transcoding method of claim 7, wherein the video uploading terminal stored in the transcoding control server specifically comprises:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
9. The distributed video transcoding method of claim 7, wherein after the transcoding plan file is issued to all the stored video uploading terminals, the method further comprises:
receiving a network connection rate of a video uploading terminal, wherein the network connection rate is a real-time network connection rate of the video uploading terminal and a video data server;
and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
10. The distributed video transcoding method of claim 7, wherein the transcoding control server further stores a performance index of the video uploading terminal, the performance index is calculated from a CPU main frequency, a memory capacity, a hard disk capacity and a network width of the video uploading terminal, and the generating of the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically comprises:
for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file;
for each video uploading terminal, calculating the ratio of the estimated total transcoding time to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
11. A distributed video transcoding device is applied to a video uploading terminal, and comprises:
the transcoding task establishing unit is used for scanning a local video file to be transcoded and establishing a transcoding task file, wherein the transcoding task file comprises the file name, the file path, the file size, the file format and the estimated transcoding time length of the video file to be transcoded;
the transcoding task uploading unit is used for uploading the transcoding task file to the transcoding control server so that the transcoding control server generates a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
the transcoding task updating unit is used for receiving the transcoding plan file issued by the transcoding control server and updating the transcoding task file according to a transcoding plan corresponding to the transcoding plan file in the transcoding plan file;
and the transcoding uploading unit is used for executing a transcoding process according to the updated transcoding task file and uploading the transcoded video file to the video data server.
12. The distributed video transcoding device of claim 11, wherein the transcoding uploading unit is specifically configured to:
and in the process of executing transcoding according to the updated transcoding task file, if at least two video files to be transcoded exist, transcoding a next video file to be transcoded in the process of uploading one transcoded video file to a video data server.
13. The distributed video transcoding device of claim 11, wherein the transcoding uploading unit is further configured to:
in the process of uploading the transcoded video file to a video data server, detecting the network connection rate of a video uploading unit and the video data server in real time, and periodically sending the network connection rate to the transcoding control server, so that the transcoding control server dynamically calculates the uploading time of a corresponding video uploading terminal.
14. The distributed video transcoding device of claim 11, further comprising:
and the pause continuous-transcoding unit is used for detecting the CPU occupancy rate, the memory occupancy rate and the hard disk capacity occupancy rate of the video uploading terminal in real time in the transcoding execution process, pausing transcoding after finishing the video file currently being transcoded when any one of the occupancy rates is greater than the corresponding occupancy rate threshold value, and continuing a subsequent transcoding task after the corresponding occupancy rate meets the preset condition.
15. The distributed video transcoding device of claim 11, further comprising:
the first exception handling unit is used for scanning a local video file to be transcoded and reestablishing a transcoding task file if a transcoding task is lost in the transcoding executing process, or acquiring a transcoding plan file from the transcoding control server and reestablishing the transcoding task file; and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reconstructed transcoding task file.
16. The distributed video transcoding device of claim 11, further comprising:
and the second exception handling unit is used for suspending uploading of the transcoded video file to the video data server and continuing to execute the transcoding process when the network is interrupted.
17. A distributed video transcoding device is applied to a transcoding control server, and comprises:
the transcoding task receiving unit is used for receiving transcoding task files uploaded by all the video uploading terminals, and the transcoding task files comprise file names, file paths, file sizes, file formats and pre-estimated transcoding duration of the video files to be transcoded;
the transcoding distribution unit is used for generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
and the transcoding plan issuing unit is used for issuing the transcoding plan file to all the stored video uploading terminals so that the video uploading terminals update own transcoding task files according to the transcoding plans corresponding to the video uploading terminals in the transcoding plan file.
18. The distributed video transcoding device of claim 17, wherein the video uploading terminal stored in the transcoding control server is specifically:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
19. The distributed video transcoding device of claim 17, further comprising:
the video uploading system comprises an uploading time dynamic estimation unit, a video data server and a video uploading terminal, wherein the uploading time dynamic estimation unit is used for receiving the network connection rate of the video uploading terminal, and the network connection rate is the real-time network connection rate of the video uploading terminal and the video data server; and dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
20. The distributed video transcoding device of claim 27, wherein the transcoding control server further stores a performance index of the video uploading terminal, the performance index is calculated from a cpu master frequency, a memory capacity, a hard disk capacity, and a network width of the video uploading terminal, and the transcoding allocating unit specifically includes:
the pre-estimated total transcoding time length unit is used for calculating and obtaining the pre-estimated total transcoding time length for transcoding the local video file to be transcoded of each video uploading terminal according to the uploaded transcoding task file;
the task load calculation unit is used for calculating the ratio of the pre-estimated total transcoding time length to the performance index for each video uploading terminal, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and the task balancing unit is used for distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
21. A distributed video transcoding system is characterized by comprising a transcoding control server and at least two video uploading terminals;
the video uploading terminal is used for scanning a local video file to be transcoded, establishing a transcoding task file and uploading the transcoding task file to the transcoding control server, wherein the transcoding task file comprises a file name, a file path, a file size, a file format and a pre-estimated transcoding time of the video file to be transcoded;
the transcoding control server is used for generating a transcoding plan file according to the transcoding task file uploaded by each video uploading terminal and sending the transcoding plan file to all the stored video uploading terminals, and the transcoding plan file comprises transcoding plans of all the video uploading terminals stored by the transcoding control server;
and the video uploading terminal is also used for updating the transcoding task file according to the transcoding plan corresponding to the video uploading terminal in the transcoding plan file, executing a transcoding process according to the updated transcoding task file, and uploading the transcoded video file to the video data server.
22. The distributed video transcoding system of claim 21, wherein the video uploading terminal stored by the transcoding control server is specifically:
the CPU main frequency is larger than a preset main frequency threshold, the memory is larger than a preset memory threshold, the hard disk capacity is larger than a preset hard disk capacity threshold, and the network broadband is larger than a preset network broadband threshold.
23. The distributed video transcoding system of claim 21, wherein the video uploading terminal is further configured to, during the transcoding process performed according to the updated transcoding task file, if there are at least two video files to be transcoded, transcode a next video file to be transcoded while uploading a transcoded video file to the video data server.
24. The distributed video transcoding system of claim 21, wherein the video uploading terminal is further configured to detect a network connection rate between the video uploading terminal and the video data server in real time during the process of uploading the transcoded video file to the video data server, and periodically send the network connection rate to the transcoding control server;
and the transcoding control server is also used for dynamically calculating the uploading time of the corresponding video uploading terminal according to the network connection rate.
25. The distributed video transcoding system of claim 21, wherein the video uploading terminal is further configured to detect a CPU occupancy rate, a memory occupancy rate, and a hard disk capacity occupancy rate of the video uploading terminal in real time during transcoding execution, and when any one of the occupancy rates is greater than a corresponding occupancy rate threshold, pause transcoding after a currently transcoded video file is completed, and continue a subsequent transcoding task after the corresponding occupancy rate meets a preset condition.
26. The distributed video transcoding system of claim 21, wherein the video uploading terminal is further configured to scan a local video file to be transcoded and reestablish a transcoding task file if a transcoding task is detected to be lost during the transcoding process, or obtain a transcoding plan file from the transcoding control server and reestablish a transcoding task file; and according to the video file list which is uploaded completely, removing the video files which are uploaded completely from the reconstructed transcoding task file.
27. The distributed video transcoding system of claim 21, wherein the video uploading terminal is further configured to suspend uploading the transcoded video file to the video data server and continue to perform the transcoding process when the network is interrupted.
28. The distributed video transcoding system of claim 21, wherein the transcoding control server is further configured to store a performance index of the video uploading terminal, where the performance index is calculated from a cpu dominant frequency, a memory capacity, a hard disk capacity, and a network width of the video uploading terminal;
the process that the transcoding control server generates the transcoding plan file according to the transcoding task file uploaded by each video uploading terminal specifically comprises the following steps:
for each video uploading terminal, calculating to obtain the estimated total transcoding time for transcoding the local video file to be transcoded according to the uploaded transcoding task file;
for each video uploading terminal, calculating the ratio of the estimated total transcoding time to the performance index, and taking a logarithmic value with the base 10 as the ratio to obtain the task load of the video uploading terminal;
and distributing part of transcoding tasks of the video uploading terminals with the task loads larger than a preset load threshold value to the video uploading terminal with the minimum task load.
29. A readable storage medium having a program stored thereon, wherein the program, when executed by a processor, performs the steps of the distributed video transcoding method of any of claims 1-6.
30. A readable storage medium having a program stored thereon, wherein the program, when executed by a processor, performs the steps of the distributed video transcoding method of any of claims 7-10.
31. An electronic device comprising a memory and a processor, the memory for storing a program; the processor is configured to execute the program to implement the steps of the distributed video transcoding method according to any one of claims 1 to 6.
32. An electronic device comprising a memory and a processor, the memory for storing a program; the processor is configured to execute the program to implement the steps of the distributed video transcoding method according to any of claims 7 to 10.
CN202010749811.7A 2020-07-30 2020-07-30 Distributed video transcoding method and related device Active CN114071149B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010749811.7A CN114071149B (en) 2020-07-30 2020-07-30 Distributed video transcoding method and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010749811.7A CN114071149B (en) 2020-07-30 2020-07-30 Distributed video transcoding method and related device

Publications (2)

Publication Number Publication Date
CN114071149A true CN114071149A (en) 2022-02-18
CN114071149B CN114071149B (en) 2023-04-11

Family

ID=80227245

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010749811.7A Active CN114071149B (en) 2020-07-30 2020-07-30 Distributed video transcoding method and related device

Country Status (1)

Country Link
CN (1) CN114071149B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116089363A (en) * 2023-04-04 2023-05-09 联宝(合肥)电子科技有限公司 Data processing method, device, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104168488A (en) * 2014-08-29 2014-11-26 北京奇艺世纪科技有限公司 Video transcoding method and device
US9565430B1 (en) * 2011-12-15 2017-02-07 KingdomSite Ministries Apparatus and method for video encoding and transcoding system
CN108712422A (en) * 2018-05-18 2018-10-26 网宿科技股份有限公司 A kind of method and apparatus creating transcoding task
CN111416992A (en) * 2020-03-31 2020-07-14 中国建设银行股份有限公司 File transcoding method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9565430B1 (en) * 2011-12-15 2017-02-07 KingdomSite Ministries Apparatus and method for video encoding and transcoding system
CN104168488A (en) * 2014-08-29 2014-11-26 北京奇艺世纪科技有限公司 Video transcoding method and device
CN108712422A (en) * 2018-05-18 2018-10-26 网宿科技股份有限公司 A kind of method and apparatus creating transcoding task
CN111416992A (en) * 2020-03-31 2020-07-14 中国建设银行股份有限公司 File transcoding method and device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116089363A (en) * 2023-04-04 2023-05-09 联宝(合肥)电子科技有限公司 Data processing method, device, electronic equipment and storage medium
CN116089363B (en) * 2023-04-04 2023-09-01 联宝(合肥)电子科技有限公司 Data processing method, device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN114071149B (en) 2023-04-11

Similar Documents

Publication Publication Date Title
CN109618002B (en) Micro-service gateway optimization method, device and storage medium
CN109905259B (en) Communication connection maintaining method, system and related equipment
CN109491797A (en) Date storage method, system and computer readable storage medium
WO2015141337A1 (en) Reception packet distribution method, queue selector, packet processing device, and recording medium
CN110417650A (en) Multilink data distribution method, device and electronic equipment
CN107171839B (en) Bandwidth flow cost control method
CN112822724B (en) Message forwarding method and device
CN111611129B (en) Performance monitoring method and device of PaaS cloud platform
CN104980472A (en) Network traffic control method and device
JP2012065314A (en) Data delivery device and data delivery system
CN105554049B (en) Distributed service amount control method and device
CN109995664B (en) Method, equipment and system for transmitting data stream
CN112787952B (en) Service flow adjusting method and device
CN114071149B (en) Distributed video transcoding method and related device
JP6468193B2 (en) Communication device, network available bandwidth estimation method in communication device, and network available bandwidth estimation program
CN103457748A (en) Data transmission method, system and device
CN102316483A (en) Method and device for ensuring quality of service (QoS) of application service in evolution-data optimized (EVDO) system
CN113885794A (en) Data access method and device based on multi-cloud storage, computer equipment and medium
CN110597631B (en) Resource management method, monitoring server, proxy server, and storage medium
CN111163079A (en) System, method, storage medium and device for distributing and controlling reported data of device
CN103179051B (en) A kind of retransmission method of Streaming Media and system
CN103051979B (en) streaming media processing method and system
JP2014036346A (en) Load balancing method in radio communication system by redirection, gateway device and user management device
WO2018045518A1 (en) Volte communication method and base station
CN108334424B (en) Method for filtering redundant data of network communication management platform based on LPWAN technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant