CN108155999B - Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism - Google Patents
Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism Download PDFInfo
- Publication number
- CN108155999B CN108155999B CN201711449497.5A CN201711449497A CN108155999B CN 108155999 B CN108155999 B CN 108155999B CN 201711449497 A CN201711449497 A CN 201711449497A CN 108155999 B CN108155999 B CN 108155999B
- Authority
- CN
- China
- Prior art keywords
- multicast
- file
- distribution
- udp
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Abstract
The invention discloses an intelligent UDP multicast file distribution system with a TCP (transmission control protocol) packet supplementing mechanism, which comprises: the multicast control end is used for correspondingly generating the multicast files into delivery plan files and then sending the delivery plan files; the multicast distributing ends are used for monitoring and acquiring the delivery plan file in real time, downloading the multicast file to an appointed directory from the multicast control end according to the analysis result after analyzing the delivery plan file, and carrying out multicast distribution on the downloaded multicast file according to a distribution mechanism; the multicast receiving terminals are used for receiving the delivery plan file sent by the multicast control terminal in real time, starting corresponding ports to be in butt joint with the multicast distributing terminals according to the analysis result and receiving the multicast file after analyzing the delivery plan file; and counting the receiving state data of the multicast file of the current multicast round in real time, uploading the receiving state data to a multicast control end and returning the scheduling result of the multicast file. The invention also discloses an intelligent UDP multicast file distribution method with the TCP packet complementing mechanism. The method of the invention can realize efficient and reliable data transmission among processes.
Description
Technical Field
The invention relates to the technical field of communication, in particular to an intelligent UDP multicast file distribution system with a TCP (transmission control protocol) packet supplementing mechanism and a method.
Background
In the prior art, most transport layers on the operating system use tcp (transmission control protocol) to implement reliable point-to-point communication. When reliable communication among a group of processes needs to be realized, the realization mode on a plurality of application layers is to establish point-to-point connection among the processes for each process to communicate with, obviously, the communication mode of the process group consumes a large amount of bandwidth and has low efficiency. UDP is a connectionless transport layer protocol in an OSI (Open System Interconnection) reference model, and although UDP multicast can implement multicast between process groups, it cannot guarantee reliable data transmission. In addition, the UDP packet is limited by the data length, and the existing UDP multicast does not belong to streaming.
Disclosure of Invention
An object of the present invention is to solve at least the above problems and to provide at least the advantages described later.
The invention also aims to provide an intelligent UDP multicast file distribution system with a TCP (transmission control protocol) packet complementing mechanism and a method thereof, which make up for the defect of low UDP multicast reliability through the TCP packet complementing mechanism and realize efficient and reliable data transmission among processes.
To achieve these objects and other advantages in accordance with the purpose of the invention, there is provided an intelligent UDP multicast file distribution system with TCP padding mechanism, comprising:
the multicast control end is used for correspondingly generating the multicast files into delivery plan files and then sending the delivery plan files;
the multicast transmitting ends are used for monitoring and acquiring the delivery plan file in real time; analyzing the delivery plan file, downloading the multicast file from the multicast control end to an appointed directory according to an analysis result, and performing multicast distribution on the downloaded multicast file according to a distribution mechanism;
the multicast receiving terminals are used for receiving the delivery plan file sent by the multicast control terminal in real time, analyzing the delivery plan file, starting corresponding ports to be in butt joint with the multicast distributing terminals according to the analysis result and receiving the multicast file; counting the receiving state data of the multicast file in the current multicast turn in real time, uploading the receiving state data to the multicast control end and returning the scheduling result of the multicast file: and the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file.
Preferably, the multicast control end further includes a human-computer interaction interface provided by the centralized distribution management software, and is configured to set the delivery plan file, where the delivery plan file includes a multicast distribution task list, a multicast plan, and UDP multicast service parameters.
Preferably, the multicast distribution task list further includes a delete multicast file operation interface, which is used to delete part of the multicast files and set the scheduling result to delete the multicast files;
the multicast distribution task list also comprises a file multicast priority operation interface which is used for setting the priority of part of multicast files so as to ensure that part of multicast files are sent in each multicast turn for multiple times of circulation in priority
Preferably, the distribution mechanism is: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters;
the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, multicast port number, bandwidth, sending start time and sending end time.
Preferably, the plurality of multicast distribution terminals further include a multicast distribution database: the method is used for writing the distribution state, the multicast turns and the time consumption of each multicast turn of the multicast distribution file in real time, wherein the distribution state of the multicast file comprises UDP data packets currently sent by a plurality of multicast distribution terminals and the number of the UDP data packets currently received by a plurality of multicast receiving terminals.
Preferably, when the predetermined multicast end time is 24h away, the multicast end time is recalculated, specifically:
counting the total number of the missing data packets of the multicast receiving terminals and the average receiving rate of each multicast after the last multicast round which is finished at present:
average receiving speed is current accumulated total packet receiving number/multicast task duration, unit: packets/s (or Bytes/s 1460 packets/s);
the newly increased multicast time is the total packet number of the missing data/the average receiving speed;
the latest end time is the preset multicast end time plus the newly-added multicast time-24 hours;
and finally, modifying the preset multicast end time into the latest end time, generating a new multicast plan file, creating multicast download tasks, and sending the multicast download tasks to the multicast receiving terminals.
Preferably, the multicast control end further includes a multicast state return interface, configured to monitor a request for uploading the current receiving state data of the multicast file received by the multiple multicast receiving terminals in real time, and write the current file receiving state data of the terminal into a multicast distribution scheduling database, where the multicast state return interface reads the distribution scheduling result of the file from the multicast distribution scheduling database and returns the distribution scheduling result to the multiple multicast receiving terminals.
Preferably, in the current multicast round, after the multicast control end starts to receive the reception status data of the terminal of the multicast file with a certain number, the new reception status data of the terminal is no longer received after an interval of at least 120s, which indicates that the return data of the multicast file in the current multicast round has been received.
Preferably, the data fixed length of the UDP data packet is 1460, the missing data threshold SR of a single multicast file is 20M, and before or after the end of the current multicast round, if the difference between the total amount of UDP data packets to be received by the multicast file and the total amount of UDP data packets actually received by the multicast receiving terminals is not greater than SR plus terminal count, the scheduling result of the multicast file is set to a padding mode, that is, the multicast file is to be switched to a TCP padding mode after the next multicast round;
wherein, the total amount of packets to be received by a single multicast file is terminal number and total UDP data packet number of the file;
the total number of actually received packets of a single multicast file is sum (the number of currently received UDP packets by each terminal);
the total number of packets of the multicast file which should be received in each multicast turn is sum (the total number of UDP data packets which should be received by a single multicast file);
the total number of packets actually received in each multicast round is sum (the total number of UDP packets actually received by a single multicast file).
An intelligent UDP multicast file distribution method with a TCP (transmission control protocol) packet complementing mechanism is characterized by comprising the following steps:
the multicast control end correspondingly generates a delivery plan file from the multicast file and then sends the delivery plan file;
a plurality of multicast transmitting terminals monitor and acquire the delivery plan file in real time;
a plurality of multicast distribution terminals analyze the delivery plan file and download the multicast file from the multicast control terminal to an appointed directory according to the analysis result, and perform multicast distribution on the downloaded multicast file according to a distribution mechanism, wherein the distribution mechanism is as follows: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters; the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, a multicast port number, bandwidth, sending start time and sending end time;
the multicast receiving terminals receive the delivery plan file sent by the multicast control terminal in real time, analyze the delivery plan file, start corresponding ports to be in butt joint with the multicast distribution terminals according to the analysis result and receive the multicast file;
a plurality of multicast receiving terminals count the receiving state data of the multicast file in the current multicast turn in real time, upload the receiving state data to the multicast control terminal and return the scheduling result of the multicast file: the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file;
and receiving a multicast control end in real time and returning a scheduling result of the multicast file, if a TCP (transmission control protocol) packet complementing mode is returned to be started, switching to the TCP packet complementing mode in the next multicast round, and synthesizing a UDP (user Datagram protocol) data packet actually received in the previous round and the TCP packet complementing mode in the current round into the multicast file after the completion of the packet complementing.
The invention at least comprises the following beneficial effects: the system of the invention makes up the defect of lower reliability of UDP multicast by a TCP packet complementing mechanism, and realizes efficient and reliable data transmission among processes;
the method specifically comprises the following steps: the method comprises the steps that a multicast control end is arranged and used for sending multicast files after delivery plan files are correspondingly generated into delivery plan files and sending the delivery plan files to a plurality of multicast distribution ends and a plurality of multicast receiving terminals in an HTTP reliable communication mode, the delivery plan files contain multicast parameters of the multicast files and return multicast state data through an interface to obtain multicast scheduling returned by a multicast database, and further multicast control over the multicast files is achieved;
the multicast file can be manually deleted and the multicast priority can be set on a human-computer interaction interface of the multicast control end so as to be sent in a circulating mode for multiple times in each multicast turn; multiple multicast tasks can be concurrent: if a new file needs to be added after a multicast task is sent out, the new multicast task can be added to complete the process;
and the multicast ending time can be dynamically or manually adjusted: and adjusting the multicast ending time according to the overall receiving state, and ensuring the efficient and accurate multicast transmission of the multicast file.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention.
Drawings
Fig. 1 is a multicast distribution data flow diagram of the intelligent UDP multicast file distribution system with TCP complementary package mechanism according to the present invention.
Detailed Description
The present invention is further described in detail below with reference to the attached drawings so that those skilled in the art can implement the invention by referring to the description text.
It will be understood that terms such as "having," "including," and "comprising," as used herein, do not preclude the presence or addition of one or more other elements or groups thereof. Preferably, the multicast control end further includes a multicast distribution server address list, which supports single selection or multiple selection to correspondingly set multiple multicast distribution ends of the multicast file.
The invention provides an intelligent UDP multicast file distribution system with a TCP (transmission control protocol) packet supplementing mechanism, which comprises: the multicast control end is used for correspondingly generating the multicast files into delivery plan files and then sending the delivery plan files; the multicast transmitting ends are used for monitoring and acquiring the delivery plan file in real time; analyzing the delivery plan file, downloading the multicast file from the multicast control end to an appointed directory according to an analysis result, and performing multicast distribution on the downloaded multicast file according to a distribution mechanism; the multicast receiving terminals are used for receiving the delivery plan file sent by the multicast control terminal in real time, analyzing the delivery plan file, starting corresponding ports to be in butt joint with the multicast distributing terminals according to the analysis result and receiving the multicast file; counting the receiving state data of the multicast file in the current multicast turn in real time, uploading the receiving state data to the multicast control end and returning the scheduling result of the multicast file: and the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file.
In the scheme, the multicast file is correspondingly generated into a delivery plan file through the multicast control end and then sent out, the delivery plan file is transmitted to the multicast distribution end and the multicast receiving ends in an HTTP reliable communication mode, the data loss situation cannot occur, the multicast distribution ends download the multicast file through an HTTP downloader according to the analyzed delivery plan file, it is guaranteed that the download is free from errors, and then UDP multicast communication connection is established between the multicast distribution ends and the multicast receiving terminals through corresponding gateways to realize UDP multicast and send the needed multicast file to the appointed multicast receiving terminals.
In a preferred embodiment, the multicast control end further includes a human-computer interaction interface provided by the centralized distribution management software, and is configured to set the delivery plan file, where the delivery plan file includes a multicast distribution task list, a multicast plan, and UDP multicast service parameters.
In a preferred embodiment, the multicast distribution task list further includes a delete multicast file operation interface, which is used to delete part of the multicast files and set the scheduling result of the delete multicast file as the delete multicast file; the multicast distribution task list also comprises a file multicast priority operation interface which is used for setting the priority of part of multicast files so as to ensure that part of multicast files are sent in each multicast turn for multiple times of circulation in priority
In a preferred embodiment, the distribution mechanism is: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters;
the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, multicast port number, bandwidth, sending start time and sending end time.
In a preferred embodiment, the multicast distribution terminals further include a multicast distribution database: the method is used for writing the distribution state, the multicast turns and the time consumption of each multicast turn of the multicast distribution file in real time, wherein the distribution state of the multicast file comprises UDP data packets currently sent by a plurality of multicast distribution terminals and the number of the UDP data packets currently received by a plurality of multicast receiving terminals.
In a preferred scheme, when the distance is 24h from the predetermined multicast end time, the multicast end time is recalculated, specifically:
counting the total number of the missing data packets of the multicast receiving terminals and the average receiving rate of each multicast after the last multicast round which is finished at present:
average receiving speed is current accumulated total packet receiving number/multicast task duration, unit: packets/s (or Bytes/s 1460 packets/s);
the newly increased multicast time is the total packet number of the missing data/the average receiving speed;
the latest end time is the preset multicast end time plus the newly-added multicast time-24 hours;
and finally, modifying the preset multicast end time into the latest end time, generating a new multicast plan file, creating multicast download tasks, and sending the multicast download tasks to the multicast receiving terminals.
In a preferred embodiment, the multicast control end further includes a multicast state backhaul interface, configured to monitor a request for uploading the reception state data of the multicast file currently received by the multiple multicast receiving terminals in real time, and write the current file reception state data of the terminal into a multicast distribution scheduling database, where the multicast state backhaul interface reads a distribution scheduling result of the file from the multicast distribution scheduling database and returns the distribution scheduling result to the multiple multicast receiving terminals.
In a preferred embodiment, in the current multicast round, after the multicast control end starts to receive the reception status data of the terminal of the multicast file with a certain number, the new reception status data of the terminal is no longer received after an interval of at least 120s, which indicates that the return data of the multicast file in the current multicast round has been received.
In a preferred scheme, the data fixed length of the UDP data packet is 1460, the missing data threshold SR of a single multicast file is 20M, and before or after the end of the current multicast round, if the difference between the total amount of the multicast file that should receive the UDP data packets and the total amount of the plurality of multicast receiving terminals that actually receive the UDP data packets is less than or equal to SR plus terminal count, the scheduling result of the multicast file is set to be a padding mode, that is, the multicast file is to be switched to a TCP padding mode after the next multicast round;
wherein, the total amount of packets to be received by a single multicast file is terminal number and total UDP data packet number of the file;
the total number of actually received packets of a single multicast file is sum (the number of currently received UDP packets by each terminal);
the total number of packets of the multicast file which should be received in each multicast turn is sum (the total number of UDP data packets which should be received by a single multicast file);
the total number of packets actually received in each multicast round is sum (the total number of UDP packets actually received by a single multicast file).
An intelligent UDP multicast file distribution method with a TCP (transmission control protocol) packet complementing mechanism is characterized by comprising the following steps:
the multicast control end correspondingly generates a delivery plan file from the multicast file and then sends the delivery plan file;
a plurality of multicast transmitting terminals monitor and acquire the delivery plan file in real time;
a plurality of multicast distribution terminals analyze the delivery plan file and download the multicast file from the multicast control terminal to an appointed directory according to the analysis result, and perform multicast distribution on the downloaded multicast file according to a distribution mechanism, wherein the distribution mechanism is as follows: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters; the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, a multicast port number, bandwidth, sending start time and sending end time;
the multicast receiving terminals receive the delivery plan file sent by the multicast control terminal in real time, analyze the delivery plan file, start corresponding ports to be in butt joint with the multicast distribution terminals according to the analysis result and receive the multicast file;
a plurality of multicast receiving terminals count the receiving state data of the multicast file in the current multicast turn in real time, upload the receiving state data to the multicast control terminal and return the scheduling result of the multicast file: the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file;
and receiving a multicast control end in real time and returning a scheduling result of the multicast file, if a TCP (transmission control protocol) packet complementing mode is returned to be started, switching to the TCP packet complementing mode in the next multicast round, and synthesizing a UDP (user Datagram protocol) data packet actually received in the previous round and the TCP packet complementing mode in the current round into the multicast file after the completion of the packet complementing.
Example 1
Firstly, a multicast control end: centralized distribution management software (hereinafter referred to as background) provides an interactive management interface (as shown in FIG. 1)
1. Business processes and logic: firstly, creating a multicast distribution server list and a multicast distribution file list; then, a multicast distribution task is established, and multicast parameters are set; issuing and activating a multicast distribution task; inquiring and monitoring multicast distribution task state
2. Multicast distribution file list: there are two sources for multicast distribution files: one is a resource file, which is derived from a resource file list (including a resource file storage path). The other is to specify other file directories, originating from directories on the file server.
File server requirements: has network read-write authority. Typical file server configuration: web + php environment.
When the multicast task has started but not ended, the list in the multicast task may be edited and the file may be deleted from the multicast distribution list. (but files cannot be added, adding files requires creating a new multicast task). After the deletion operation is executed, the file scheduling result in the multicast task plan database needs to be set to 2.
Supplementing: because the original multicast distribution file list is not friendly to operation, the creation of the multicast distribution file list is changed into the following two modes:
a multicast distribution file list creation mode; modifying and perfecting the creating service logic of the current multicast distribution file list:
description of the drawings: changing the 'newly added' button into 'newly added media resource file list';
changing 'list of generated files' into 'list of newly added other directory files'
The addition of the other directory file lists can be realized by adding the other directory file lists on the interactive management interface, and the specific description is as follows:
1) the "preferred number of files" field value is set to 1;
2) clicking a 'select file' button to list all directories and files under a source file path;
3) after the user selects the files, counting the total number of the selected files; compressing the corresponding file into a compressed packet corresponding to information such as the file path and the like; the compressed packet is stored in the source file path; [ preserving the storage route of compressed bags ]
4) When the file list of the other files is established, the other file directory file lists are established;
5) adding an input item in the list of the added other directory files: and decompressing the command. Is a pull-down option. There is currently only one option.
3. When the multicast task is created in an m _ group _ distribution _ task _ php model;
1) creating a multicast task, clicking a button for issuing the multicast task, ① generating a json file of a multicast task plan, writing a decompression command into the json file, ② generating a cache task and a cache task detail of the json file, ③ generating the cache task and the cache task detail of a compressed packet;
2) when creating a multicast task and generating a cache task, a gateway packet is to be distinguished here, whether the gateway packet is a multicast distribution server packet or a multicast reception packet [ m _ gateway _ group.php can be distinguished when creating a packet ], and a json file task issues an object: multicast distribution server grouping and multicast reception grouping; and (3) compressing the cache task issuing object of the packet: grouping the multicast distribution servers;
3) so far, the compression packet tasks of the json file and other directory files are completely issued;
4. multicast tasks and gateway grouping: one-to-many relationship
1) In the multicast distribution task model, gateway grouping is changed to support multiple selection;
3) the 'task number' field changes the numbering rule to renumber from 00000001 and increases;
3) adding a field of 'cache task number', and a numbering rule of 'cache task number': selecting several groups to generate several numbers; the number of the cache task is numbered from the number of the cache task (m _ cache _ task.php), and the number is increased;
4) generating corresponding cache tasks and cache task details by using cache task numbers in the multicast distribution tasks, wherein the cache tasks comprise json files and compressed packet files;
the modification is that the test platform is developed and tested on go3c, and is deployed on Aliyun after the test is passed.
5. Modifying media asset file multicast distribution file list service logic
Firstly, the new media asset file import is completed by using an original media asset file model, and the description is as follows:
1) a batch import function is required to be added, and an xls media asset file list with a specified format is imported;
2) adding a checking function of the media asset files in the existing media asset file model; and after the check, providing optional operations: "generate multicast distribution file list" and "generate cache task detail".
3) In the existing media asset file model, data items of media asset files are added: providers of
4) Add filter and search terms: screening can be performed based on the target path, the download path, the target file name and the file type; and can search;
5) selecting 'generating a multicast distribution file list', and entering a new item of the multicast distribution file list with the checking file;
6) and if the generation of the cache task detail is selected, the newly-added cache task detail of the newly-added cache task with the checking file is entered.
Two, a plurality of multicast distributing terminals:
1. multicast distribution task management
Data item:
multicast distribution plan file: the default is created according to schedule, multicast task id and current time, and can be edited.
Grouping a gateway: the gateway group corresponding to the multicast distribution task can be selected more. And selecting from the existing gateway grouping, wherein only the gateway terminal under the selected gateway grouping can receive the multicast distribution task.
Multicast distribution file list: and selecting from the edited multicast distribution file list, and supporting single selection and multiple selection.
The multicast distribution server: and selecting from a multicast distribution server list (IP address or domain name) recorded in a background, and supporting single selection and multiple selection. For example: 202.193.1.24
Multicast address + port number: the multicast address and port number of the task are specified and should have uniqueness.
Multicast bandwidth: setting the bandwidth of the multicast task, and manually filling the bandwidth by taking K as a unit.
Start time: the start time of the multicast task is '2017-03-2912: 00: 00'
End time: the end time of the multicast task is '2017-04-2923: 59: 59'.
N (fixed length of multicast distribution packet file data): 1460
SR (single file missing data threshold): and manually filling in the unit of M. Typical values: 20M
The circulation times of the priority files are as follows: manual fill, typical value 5, span: 1-256
The terminal receives the state and returns url, public network IP address, which can be domain name.
The operation is as follows: add, edit, delete, manually start, close, pause.
And (3) deleting: directly deleting the multicast tasks which are not issued and finished; waiting for the started multicast task and deleting the cache task;
in the multicast and in the suspended multicast task, setting all file scheduling state values in the multicast task to be 2 (namely deleting);
pausing: and only the task in the multicast is effective, and the scheduling state values of all files in the multicast task are set to be 3 (a new value, 3 is suspended).
2. Multicast task status query
Data item
And (3) finishing the downloading task of the UDP multicast distribution server: same as original downloader
The multicast distribution state of the UDP multicast distribution server is as follows:
multicast turns: showing that the current run is the number of rounds,
the multicast file is completed in the current round:
the file is being multicast:
actual multicast bandwidth:
n (effective packet fixed length): 1460
SR (single file missing data threshold): 20M
Time consumption of multicast in the previous round: hours
Status of each video file received by the terminal UDP:
file name:
size:
multicast turns: and extracting a scheduling database on the UDP multicast distribution server.
Percent completion: as above.
Multicast distribution file reception details
The multicast distribution file receiving detail model is used for displaying and inquiring the receiving state of each file of each gateway.
The multicast receiving file state display logic is as follows:
the receiving state of each file of each terminal is obtained from a database (in a 2.5.1 interface) and is displayed in combination with a scheduling result
In multicasting; the file receiving is not finished, and the corresponding scheduling result is 0;
in the bag supplement: the file receiving is not finished, and the corresponding scheduling result is 1;
and (3) deleting: the corresponding scheduling result is 2;
pausing: the corresponding scheduling result is 3;
and (3) completing: the file reception is completed.
3. Multicast distribution background program
1) Multicast distribution file download sequence generation and transmission
After the multicast task is issued, a multicast distribution file downloading sequence is generated at the same time for the multicast distribution server to download. The business logic is as follows:
and (3) media resource file list processing: and directly selecting a multicast distribution server group for each media asset file in the media asset file list, and creating a cache task id and a cache task detail. And the multicast distribution server downloads the data through a downloader.
List of other directory files: and creating a caching task id and a caching task detail for the compressed package file of the directory file list, so that the multicast distribution server can download the caching task details through a downloader.
2) Multicast distribution plan file generation and transmission
Multicast distribution plan file format definition
File naming: schedule + id + current time txt (or json), for example: schedule0001-20170107141000.txt
One multicast task may change the multicast end time, so there are multiple multicast distribution plan files. The id of the multicast distribution plan file of the same multicast task must be the same.
Data format:
{
id1, amounts1, bburl1, id2, amounts2, bburl2\ n// gateway grouping, total number of terminals under the gateway grouping, and supplementary packaging url
234.1.2.3:11111\ n// multicast address + port number
Uniline, luxspace, com \ n// terminal receiving state back-transmitting url
2017-03-2917: 00:00\ n// start time
2017-04-1500: 00:00\ n// end time
1000000,3000,1460,2,20,3\ n// this multicast bandwidth, total number of distributed files 3000, effective data fixed packet length 1460, SR value 2M, first 20 files are priority files, and are repeatedly played for 3 times in each turn.
Data/media/vodfiles/\ n// File deposit Path
1, am 4,980126213,675950,45d15629d2ad1cb77a055c28f3954445, tarxvf \ n
v/File number 1, File name Argan Zhengchuan mp4, File size 980126213, Total Package number 675950, md5
45d15629d2ad1cb77a055c28f3954445, file operation instruction string
Subdirectories are contained in the/data/media/volfiles/sub 1/\ n// file storage path
103, 001.mp4,120000000,675950,45d15629d2ad1cb77a055c28f3954445\ n
.........
// so on for 3000 file data
}
Example (c):
{"bburl":"192.168.1.150:1088","groups":[{"id":"00002.0015","cnt":"3"}],"addr":"234.1.2.3","port":"10000","url":"120.27.138.55:8060","start":"2017-05-04 11:25:59","end":"2017-05-04 14:05:46","len":"1440","SR":"2","repeat":"3","count":"12","prority":"4","medias":[{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000001","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c1\u5b63-01.mp4","size":"136898926","md5":"","f_fileid_id":"681","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000003","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c1\u5b63-03.mp4","size":"136870959","md5":"","f_fileid_id":"683","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000004","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c1\u5b63-04.mp4","size":"136179764","md5":"","f_fileid_id":"684","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000002","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-02.mp4","size":"115733416","md5":"","f_fileid_id":"688","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000003","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-03.mp4","size":"137483356","md5":"","f_fileid_id":"689","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000005","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-05.mp4","size":"137155455","md5":"","f_fileid_id":"691","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000007","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-07.mp4","size":"137786165","md5":"","f_fileid_id":"693","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000006","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-01.mp4","size":"115102142","md5":"","f_fileid_id":"1037","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000009","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-04.mp4","size":"137379815","md5":"","f_fileid_id":"1040","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000011","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-06.mp4","size":"137511988","md5":"","f_fileid_id":"1042","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000013","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-08.mp4","size":"137314297","md5":"","f_fileid_id":"1044","pkgs":"1"},{"path":"\/data\/media\/vodfiles\/001\/","Id":"00000014","name":"\u547c\u53eb\u52a9\u4ea7\u58eb\u7b2c2\u5b63-09.mp4","size":"136776511","md5":"","f_fileid_id":"1045","pkgs":"1"}]}
file generation
File generation logic:
and when creating and storing the multicast distribution task, the background generates a multicast distribution plan file according to the format calculation.
If a plurality of media asset file distribution lists exist in the multicast distribution task, the number of the preferred files and the cycle number are obtained from the first media asset file distribution list; if there are only other directory file lists, the preferred number of files is set to 1 (i.e., the number of compressed packet files), and the number of cycles is also set to 1.
And after editing time in the multicast distribution process, storing a new distribution plan file which has the same id but different time stamps.
And after the multicast distribution scheduling program dynamically adjusts the end time, generating a new distribution plan file with the same id but different timestamps.
File delivery
And creating a plurality of cache tasks for the multicast distribution plan file, and respectively corresponding to the multicast distribution server group and all receiving terminal groups, so that the multicast distribution server group and all receiving terminal groups in the multicast distribution task receive the multicast distribution plan file through a downloader.
4) Multicast distribution statistics and scheduling
Business logic
Counting: after the multicast distribution task starts, a checkmdstatus interface is called every checkmsdtime (unit s, default value 60) to inquire a multicast distribution service state interface, and the acquired parameters are written into a multicast distribution database.
And meanwhile, reading the database in real time, and counting the number of packets currently received by each terminal of the transmitted files (judged by sendflag that 1 is transmitted and 0 is not transmitted) in the current multicast turn.
Total number of packets to be received by single file is terminal number and total number of packets of file
The total number of packets actually received in a single file is sum (the number of packets currently received in each terminal)
The total number of packets to receive a file in each multicast round is sum (the total number of packets to receive a single file)
The total number of packets actually received in each multicast round equals sum (total number of packets actually received in a single file)
Time-consuming per multicast round (unit: hour): and calling a multicast distribution state interface to obtain and keeping historical data.
Average receiving speed is current accumulated total packet receiving number/multicast task duration, unit: bags/s (or Bytes/s 1460 packs/s)
Scheduling: according to the statistical result, the multicast distribution scheduling is carried out on the file in real time based on the following strategies:
and if the difference value between the total amount to be received of the file and the total amount actually received of the terminal is less than or equal to SR, setting the scheduling result of the file to be 1, namely, indicating that the file is switched to a TCP (transmission control protocol) packet supplementing mode after the next multicast turn.
Counting the completed last multicast round when the multicast task is finished by a checkpoint time (default 24 hours), and recalculating the multicast finish time by the total number of the missing data packets of all the terminals and the average receiving rate of each multicast round:
new multicast time (total number of missing data packets/average receiving speed)
The latest ending time is the last ending time plus the newly added multicast time-24 hours
And then, modifying the multicast ending time, generating a new multicast plan file, creating a download task, and sending the download task to all target terminals.
(note: when the number of gateways under a gateway packet changes, the statistics and scheduling also need to be adjusted.)
4. Terminal receiving state return interface implementation
A multicast distribution protocol is added based on the file transfer synchronization interface modification of the previous downloader (HTTP downloader).
And monitoring the real-time uploading request of the current file receiving state data of all the terminals in real time, and writing the current file receiving state data of the terminals into a multicast distribution scheduling database.
In the current multicast turn, if the terminal which starts to receive a certain numbered file receives the state data, after no new terminal state data is received any more at a time interval (tdURTIME, one second, default value 120s), setting sendts of the file to be 1; indicating that the return data of the file in the multicast turn has been received.
And the interface simultaneously reads the distribution scheduling result of the file from the database and returns the distribution scheduling result to the terminal.
The scheduling result is: 0: continuing multicast reception; 1 starting a TCP packet complementing mode; 2 delete the file.
5. Multicast distribution scheduling interface implementation
And monitoring the file distribution scheduling request of the multicast distribution server in real time, and returning the scheduling result to the multicast distribution server.
6. Multicast distribution database design
Global parameters
Multicast delivery server address: and selecting from a multicast distribution server list (IP address or domain name) recorded in a background, and supporting single selection and multiple selection. For example: 202.193.1.24
Multicast address + port number: the multicast address and port number of the task are specified and should have uniqueness.
Multicast bandwidth: setting the bandwidth of the multicast task, and manually filling the bandwidth by taking K as a unit.
Start time: the start time of the multicast task is '2017-03-2912: 00: 00'
End time: the end time of the multicast task is '2017-04-2923: 59: 59'.
N (fixed length of multicast distribution packet file data): 1460
SR (single file missing data threshold): and manually filling in the unit of M. Typical values: 20M
The circulation times of the priority files are as follows: manual fill, typical value 5, span: 1-256
Multicast reception status return address url:
multicast distribution service status database:
last multicast round is time consuming
Current multicast round
Current multicast round start time
Current sending file number and udp packet number
Real-time transmission bandwidth
Multicast distribution scheduling database
Multicast round
File numbering
Sending status
Gateway numbering
Current file received packet accumulation
Missing packet number sequence
Total number of current file packs
Completion status
Scheduling results
Average receiving speed
UDP multicast distribution software
1) Multicast distribution file download
And downloading the UDP multicast distribution file by adopting an original gateway downloader.
(Note: including file transmission synchronous interface)
2) Multicast distribution master schedule
Business logic
Initialization: and reading and analyzing the multicast distribution plan file in the appointed directory, setting multicast distribution parameters, and determining whether the multicast distribution file is downloaded completely. If the multicast task is completed, a directory is created by using the multicast task name, for example: \\ task \ mtask 201703291208.
And create the following subdirectories under that directory:
directory of multi cast// multicast files
V, file storage after completing the multicast task and simultaneously used for acquiring a supplementary package.
Then, renumbering the file to be distributed and moving the file to \ multicast \ and starting or closing the multicast distribution broadcasting module according to the distribution plan;
meanwhile, a distribution scheduling database is initialized;
starting a multicast distribution return interface;
the detection confirms that the web publishing function is normal.
Scheduling
And in each multicast turn, sending the files according to the sequence of the file numbers.
Before sending the file each time, inquiring the file scheduling result to a background:
if the multicast number is 0, continuing the multicast;
if the number of the files is 1, the multicast broadcast directory is moved out after the files are sent, and multicast is not carried out;
if the number is 2, directly moving the file out of the multicast broadcast directory;
if 3, suspending all multicast; and continuing to poll the file state until the state of one file is not 3, and then recovering the multicast. For example, assuming that the current multicast round is 8, and at this time, it is detected that the scheduling result of the 358-numbered file is 1, after the 9 th round is finished, the 358-numbered file is moved from \ multicast \ to the multicast task completion directory \ finish \.
3) Multicast distribution file broadcast
UDP multicast distribution packet protocol
{
Bag identification: 0x50,1 byte
Multicast turns: 2 bytes, indicating the current multicast round. The value range is 1 to 65535
And (3) numbering files: 2 bytes, and the value range is 1-65536. The file number in the multicast task
Packet number: 4 bytes, sequence number of current packet in this document
Reserved bytes: 1 byte
The package contains the file data size: 2 bytes. The value range is 0-N. When the number is 0, it means that all the included data is file data, and when n is non-zero, it means that the first n bytes are file data.
File data: n bytes (N is fixed length of multicast distribution packet, set by background, default is 1460)
CRC 32: 4 bytes
}
When the file multicast distribution is performed, the file is divided into UDP packets with fixed length, wherein the packet length is (1+1+2+4+2+2+ N + 4).
Implementing logic
Initialization: setting parameters such as multicast ip, port number, bandwidth, sending start time, sending end time, source file path, total number of sent files, number of priority files, repeated playing times of priority files and the like
The sub-package realizes the logic:
reading a file, creating a packet header and file data according to a distribution packet protocol format, and forming a UDP file data packet with a unique identifier;
UDP multicast transmission: firstly, judging whether the data packets are in the effective sending time, if so, calculating the number of the data packets to be sent per second according to the bandwidth, and then calling a socket multicast function to send the udp packets.
File logic: and firstly, performing small-cycle transmission of the priority file. Then, a multicast major loop is performed.
Outputting parameters: current multicast round, file number, udp package number
4) http repacking module
When a certain file is moved to the \ finish \ directory by the main scheduling program and the web service is started, the file can provide a package supplementing service for the terminal.
Three multicast receiving terminals: terminal receiving software
1. Business logic
1) Creating working path and environment, analyzing delivery plan file
Firstly, establishing a working directory multicastrechv; after receiving delivery plan file schedule-20170107141000.txt (or json) through the original gateway communication interface, parameters such as multicast address and port number, movie file name, file number, md5 value, delivery time plan, multicast receiving state return address and the like issued by the movie file are analyzed. Creating a temporary receiving subdirectory for each file number: \ File number \, and create an index file under the directory: file number index. For example: when the number of the files analyzed in the delivery plan file to be the Ashland normal transfer file is 1 and the total package number is 675950, creating a subdirectory \1\ and creating 1.index under the directory.
(Note: because the multicast receive file should also be written to the local database because the original downloader only parsed the original download task)
2) And receiving the UDP packet, the group tmp file and the index file.
Then, starting a UDP receiving module, receiving the corresponding multicast address and port number, and composing the received UDP data packets into files according to the following rules:
the method comprises the steps of firstly analyzing a packet identifier, extracting a multicast turn, a file number (such as 3008) and a packet sequence number (such as 3569), determining the data size of a packet file, and then judging whether a tmp file corresponding to the file number exists or not, if not, creating. If so, writing.
And when the file number in the data packet is detected to be changed, the multicast of the current file is finished, and the next file is received. At this time, the total number of received packets in the file is counted and written into the file number index file (e.g. 3008.index), and the sequence of packet numbers (e.g. 98000,53,105,348.. times.) that are still missing after the current multicast round is recorded.
3) Real-time feedback of current file receiving state
And calling a return and packet supplementing module, and uploading the number of the received packets of the current received file plus the multicast task id, the gateway number, the current multicast round and the file number to a multicast distribution server. For example: {0001,0000013.00001,6,3008,98000}.
4) Document integrity determination and processing
After uploading, judging the integrity of the file: if the total number of the currently received packages of the file is the same as the actual total number of packages of the file, indicating that the file is received well, the tmp file of the file number is combined into the actual movie file (with MD5 check option), and the actual movie file is moved to the directory specified by the delivery plan file.
If the current received file has missing packets, monitoring the return interface, and if the data returned by the return interface is 0, indicating that the multicast reception is continuously waited. If 1, it indicates that the complementary packet receiving mode is started.
5) Complementary packet reception
After entering the patch package receiving, the patch package address (for example, \203.19.10.4\ packetfil) acquired from the delivery plan file is added with the current file number to initiate a patch package request. For example: http \ \203.19.10.4\ packetfil \ 3008. And then analyzing the file, generating a corresponding tmp file from the missing package data to a subdirectory, and executing complete judgment and processing of the above file.
6) Processing after the end of the multicast time
When the multicast ending time is up, the delivery plan file is firstly detected and updated to see whether the ending time is updated or not. If the multicast has the update, the multicast is updated to the new end time, and the multicast is continuously received.
In addition, if the multicast packet is not received for a long time (10 minutes) during the multicast reception effective time, the delivery plan file is also detected and updated. If there is an update, the update is a new end time.
After the end time confirmed as the latest is up, the above file integrity judgment and processing will be performed for all the temporary reception subdirectories.
2. Interface
1) File transmission synchronous interface (php)
A provider: background of the invention
The calling party: terminal reception program
Interface description: and a multicast distribution protocol is added by utilizing a file transmission synchronous interface of the original downloader.
And after the terminal receiving program receives the current file through udp multicast, calling the interface, and returning the receiving state of the current file of the terminal to the background in real time. In addition, after the terminal starts a TCP packet supplementing mode and completes the packet supplementing work, the interface is also called to notify the background of the completion state.
And the background monitors the real-time uploading request of the current multicast receiving file receiving state data of all the terminals in real time and writes the current file receiving state data of the terminals into a multicast distribution scheduling database.
In the current multicast turn, if a terminal which starts to receive a certain numbered file receives state data and no more receives new terminal state data at a time interval (tdurtime, unit second, default value 120s), setting a sendts parameter in the file database to be 1; (indicating that the return data of the file has been received in this multicast round.)
And the background simultaneously reads the distribution scheduling result of the file from the database and returns the result to the terminal.
The scheduling result is: 0: continuing multicast reception; 1 starting TCP complementary package mode.
Interface address: http://120.27.138.55:8060/pivosboss/api/php/php _ multicast _ report
The request mode comprises the following steps: post
The content is as follows: param ═
'{"f_id":"00000066","f_uuid":"0000013.","round":"2","f_file_id":"633","received_package":"5"}'；
The upload data adopts a json format:
{ multicast task id, gateway number, multicast round, file number, received packet number }
E.g., {0001,0000013.01001,2,3002,30870,1,5,60060
Description of the drawings: and the multicast task 0001, the gateway with the number of 0000013.01001, and after the 2 nd wheeling is finished, the currently received file with the number of 3002 has received 30870 packets.
If the packet is called when the packet is completed, the multicast round value is set to 0.
And returning an interface: 0. 1,2 and 3.0 indicates to continue multicast reception; 1 indicates to start the complementary packet reception; 2 denotes deleting the file; 3 indicates that the multicast end has paused multicasting the file.
2) Multicast distribution scheduling interface implementation
A provider: background of the invention
The calling party: multicast distribution program
Interface description: and the multicast distribution scheduling program periodically polls according to the file number in each multicast turn, calls the interface and acquires the scheduling result of the file.
Interface address: http://120.27.138.55:8060/pivosboss/api/php/php _ distribution _ schedule
The request mode comprises the following steps: post
The upload data adopts a json format: { multicast distribution server UUID, multicast task id, multicast round, File number }, e.g. {000038.0001,0001,2,3002}
Description of the drawings: multicast task 0001, multicast in 2 nd round, file numbering
And returning an interface: 0. 1,2 and 3.0 represents the multicast is continued; 1 represents stopping multicast; 2 denotes deleting the file; and 3 denotes pausing the multicast of the file.
3) Multicast distribution broadcast state query interface
A provider: background of the invention
The calling party: multicast distribution program
Interface description: the multicast distribution server reports the current sending state of the multicast distribution server periodically (which can be set as per second by default). And after the background receives the data, writing the data into the database for the multicast distribution server to inquire and call the state.
Interface address: http://120.27.138.55:8060/pivosboss/api/php/php _ distribution _ status
The request mode comprises the following steps: post
The upload data adopts a json format:
{ multicast distribution server UUID, multicast task id, current multicast round, number of files sent, current packet number sent, actual multicast bandwidth, multicast time (unit: s) in the previous round, start time of the current round }
For example {000038.0001,0001,3,24,58,890,989000,29808, 2017-04-2417: 23:34}
And returning an interface: 0 or 1. 0 failed and 1 succeeded.
While embodiments of the invention have been described above, it is not intended to be limited to the details shown, or described, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. An intelligent UDP multicast file distribution system with a TCP (transmission control protocol) packet complementing mechanism is characterized by comprising:
the multicast control end is used for correspondingly generating the multicast files into delivery plan files and then sending the delivery plan files;
the multicast transmitting ends are used for monitoring and acquiring the delivery plan file in real time; analyzing the delivery plan file, downloading the multicast file from the multicast control end to an appointed directory according to an analysis result, and performing multicast distribution on the downloaded multicast file according to a distribution mechanism;
the multicast receiving terminals are used for receiving the delivery plan file sent by the multicast control terminal in real time, analyzing the delivery plan file, starting corresponding ports to be in butt joint with the multicast distributing terminals according to the analysis result and receiving the multicast file; counting the receiving state data of the multicast file in the current multicast turn in real time, uploading the receiving state data to the multicast control end, and receiving the scheduling result of the multicast file returned by the multicast control end: the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file;
the multicast control end comprises a man-machine interaction interface provided by centralized distribution management software and is used for setting the delivery plan file, and the delivery plan file comprises a multicast distribution task list, a multicast plan and UDP multicast service parameters;
the distribution mechanism is as follows: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters;
the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, multicast port number, bandwidth, sending start time and sending end time.
2. The intelligent UDP multicast file distribution system with TCP padding mechanism of claim 1, wherein the multicast distribution task list includes a delete multicast file operation interface for deleting a part of the multicast files and simultaneously setting the scheduling result thereof to delete the multicast files;
the multicast distribution task list also comprises a file multicast priority operation interface which is used for setting the priority of part of multicast files so as to ensure that part of multicast files are sent circularly for a plurality of times in each multicast turn preferentially.
3. The intelligent UDP multicast file distribution system with TCP padding mechanism of claim 1, wherein the plurality of multicast distribution ends comprises a multicast distribution database: the method is used for writing the distribution state, the multicast turns and the time consumption of each multicast turn of the multicast distribution file in real time, wherein the distribution state of the multicast file comprises UDP data packets currently sent by a plurality of multicast distribution terminals and the number of the UDP data packets currently received by a plurality of multicast receiving terminals.
4. The intelligent UDP multicast file distribution system with TCP padding mechanism of claim 3, wherein when 24h away from the predetermined multicast end time, the multicast end time is recalculated, specifically:
counting the total number of the missing data packets of the multicast receiving terminals and the average receiving rate of each multicast after the last multicast round which is finished at present:
average receiving speed is current accumulated total packet receiving number/multicast task duration, unit: package/s;
the newly increased multicast time is the total packet number of the missing data/the average receiving speed;
the latest end time is the preset multicast end time plus the newly-added multicast time-24 hours;
and finally, modifying the preset multicast end time into the latest end time, generating a new multicast plan file, creating multicast download tasks, and sending the multicast download tasks to the multicast receiving terminals.
5. The system according to claim 1, wherein the multicast control end further includes a multicast status backhaul interface, configured to monitor a real-time upload request of the reception status data of the multicast file currently received by the multicast receiving terminals in real time, and write the current file reception status data of the terminal into the multicast distribution scheduling database, and the multicast status backhaul interface reads the distribution scheduling result of the file from the multicast distribution scheduling database and returns the distribution scheduling result to the multicast receiving terminals.
6. The system according to claim 5, wherein in the current multicast round, if the multicast controller starts to receive the receiving status data of the terminal of a multicast file with a certain number, the multicast controller does not receive new receiving status data of the terminal any more after an interval of at least 120s, which indicates that the backhaul data of the multicast file in the current multicast round has been received completely.
7. The intelligent UDP multicast file distribution system with a TCP padding mechanism according to claim 3, wherein the data fixed length of the UDP packet is 1460, the threshold value SR of missing data of a single multicast file is 20M, and before or after the current multicast round is finished, if a difference between a total amount of UDP packets to be received by the multicast file and a total amount of UDP packets actually received by the multicast receiving terminals is less than or equal to SR × total number of terminals, the scheduling result of the multicast file is set to a padding mode, that is, the multicast file is to be switched to the TCP padding mode after the next multicast round;
wherein, the total amount of packets to be received by a single multicast file is terminal number and total UDP data packet number of the file;
the total number of actually received packets of a single multicast file is sum (the number of currently received UDP packets by each terminal);
the total number of packets of the multicast file which should be received in each multicast turn is sum (the total number of UDP data packets which should be received by a single multicast file);
the total number of packets actually received in each multicast round is sum (the total number of UDP packets actually received by a single multicast file).
8. An intelligent UDP multicast file distribution method with a TCP (transmission control protocol) packet complementing mechanism is characterized by comprising the following steps:
the multicast control end correspondingly generates a delivery plan file from the multicast file and then sends the delivery plan file, wherein the delivery plan file comprises a multicast distribution task list, a multicast plan and UDP multicast service parameters;
a plurality of multicast transmitting terminals monitor and acquire the delivery plan file in real time;
a plurality of multicast distribution terminals analyze the delivery plan file and download the multicast file from the multicast control terminal to an appointed directory according to the analysis result, and perform multicast distribution on the downloaded multicast file according to a distribution mechanism, wherein the distribution mechanism is as follows: under the appointed directory, reading files according to the multicast plan sequence, converting the files into UDP data packets with unique identifiers according to a protocol format, and then carrying out multicast distribution according to set UDP multicast parameters; the UDP data packet format comprises a broadcast sequence number, a multicast file number, a UDP data packet sequence number, a UDP data packet effective data size and effective data; the UDP multicast parameters comprise multicast ip, a multicast port number, bandwidth, sending start time and sending end time;
the multicast receiving terminals receive the delivery plan file sent by the multicast control terminal in real time, analyze the delivery plan file, start corresponding ports to be in butt joint with the multicast distribution terminals according to the analysis result and receive the multicast file;
a plurality of multicast receiving terminals count the receiving state data of the multicast file in the current multicast round in real time, upload the receiving state data to the multicast control terminal and receive the scheduling result of the multicast file returned by the multicast control terminal: the scheduling result is to continue multicast reception, start a TCP packet complementing mode or delete the multicast file;
and a plurality of multicast receiving ends receive the scheduling result of the multicast file returned by the multicast control end in real time, if the TCP supplementary package mode is started, the next multicast round is switched to the TCP supplementary package mode, and the UDP data package actually received in the previous round and the TCP supplementary package of the current round are combined into the multicast file after the supplementary package is finished.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711449497.5A CN108155999B (en) | 2017-12-27 | 2017-12-27 | Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711449497.5A CN108155999B (en) | 2017-12-27 | 2017-12-27 | Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108155999A CN108155999A (en) | 2018-06-12 |
| CN108155999B true CN108155999B (en) | 2020-08-07 |
Family
ID=62463452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711449497.5A Active CN108155999B (en) | 2017-12-27 | 2017-12-27 | Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108155999B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108933835A (en) * | 2018-07-23 | 2018-12-04 | 安徽广行领视通信科技有限公司 | A kind of CDN distribution method for saving bandwidth resources |
| JP7097772B2 (en) * | 2018-07-25 | 2022-07-08 | キヤノン株式会社 | Information processing equipment, control methods and programs |
| CN111092741B (en) * | 2018-10-24 | 2022-04-29 | 玲珑视界科技（北京）有限公司 | System and method for distributing files through multicast channel |
| CN111372103B (en) * | 2018-12-26 | 2023-05-26 | 中兴通讯股份有限公司 | Multicast method, device, equipment and computer storage medium |
| CN112235406A (en) * | 2020-10-15 | 2021-01-15 | 儒安物联科技集团有限公司 | Subnet equipment information synchronization method based on multicast and memory backup |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105072508A (en) * | 2015-08-12 | 2015-11-18 | 苏州华启智能科技有限公司 | System and method for making up packets for multimedia playing of wireless network |
| CN105376165A (en) * | 2015-10-15 | 2016-03-02 | 深圳市金证科技股份有限公司 | UDP multicast method, system, sending device, and receiving device |
| CN105827588A (en) * | 2015-12-23 | 2016-08-03 | 广东亿迅科技有限公司 | Network driver layer-based streaming media data distribution system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102480518A (en) * | 2010-11-30 | 2012-05-30 | 英业达集团(天津)电子技术有限公司 | Multicast download method of mapping file |
-
2017
- 2017-12-27 CN CN201711449497.5A patent/CN108155999B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105072508A (en) * | 2015-08-12 | 2015-11-18 | 苏州华启智能科技有限公司 | System and method for making up packets for multimedia playing of wireless network |
| CN105376165A (en) * | 2015-10-15 | 2016-03-02 | 深圳市金证科技股份有限公司 | UDP multicast method, system, sending device, and receiving device |
| CN105827588A (en) * | 2015-12-23 | 2016-08-03 | 广东亿迅科技有限公司 | Network driver layer-based streaming media data distribution system |
Non-Patent Citations (1)
| Title |
|---|
| "基于NGB演示环境的电影数字拷贝传输***设计与实现";王春梅;《数字技术》;20141231;33页3.1节-35页3.2节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108155999A (en) | 2018-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108155999B (en) | Intelligent UDP multicast file distribution system and method with TCP (transmission control protocol) packet supplementing mechanism | |
| CN106685942B (en) | Video live broadcast playback system and video live broadcast playback method | |
| CN103002274B (en) | A kind of mobile multimedia real-time transcoding Play System and method of downloading based on off-line | |
| CN101895562B (en) | Mobile multimedia broadcast (CMMB) channel-based data and audio/video pushing system | |
| CN100556129C (en) | A kind of equity connects flow medium live system and device | |
| US7356029B2 (en) | IP data encapsulation and insertion in a transport multiplexer | |
| CN101785255B (en) | For the data flow con-trol of the network equipment | |
| US8429263B2 (en) | File content distribution method, device, and system for an interactive network television system | |
| US20140165119A1 (en) | Offline download method, multimedia file download method and system thereof | |
| CN111385112B (en) | Slice resource deployment method, device, slice manager and computer storage medium | |
| CN102137163B (en) | A kind of method of multimedia file shared system and shared multimedia file thereof | |
| CN102547478B (en) | Triggered slice on-demand system and method of streaming media based on CDN (Content Distribution Network) | |
| CN109688222A (en) | The dispatching method of shared computing resource, shared computing system, server and storage medium | |
| CN108989885A (en) | Video file trans-coding system, dividing method, code-transferring method and device | |
| CN104092646A (en) | Operation synchronization method and device | |
| CN102255942A (en) | Multimedia quantum flow and cloud processing technology and realization method | |
| CN104618738B (en) | A kind of smart television program broadcasting method and device | |
| CN103581764B (en) | Downloading and storing method, downloading and storing system and playing system suitable for HLS video file | |
| CN103024081A (en) | Peer-to-peer communication terminal dispatching method adaptable to time-effect-guaranteed communication systems | |
| CN108234190A (en) | A kind of management method and system regarding networked devices | |
| CN109918104A (en) | A kind of automatic updating system and method | |
| CN110445723A (en) | A kind of network data dispatching method and fringe node | |
| CN110545206A (en) | method, device and system for generating scheduling table in time-triggered network | |
| CN113301096A (en) | Method, system and node equipment for data transmission between nodes in content distribution network | |
| EP2439890A1 (en) | Method and system for processing mobile multimedia data broadcasting service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200528 Address after: 210042 floor 3, No. 699-1, Xuanwu Avenue, Xuanwu District, Nanjing City, Jiangsu Province Applicant after: Lezi Technology (Nanjing) Co., Ltd Address before: 100195, No. 3, unit 2, building 1002, two district, Changqing Garden, Beijing, Haidian District Applicant before: Luo Jianping |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |