WO2017024805A1 - 一种文件分发方法、装置和*** - Google Patents

一种文件分发方法、装置和*** Download PDF

Info

Publication number
WO2017024805A1
WO2017024805A1 PCT/CN2016/078481 CN2016078481W WO2017024805A1 WO 2017024805 A1 WO2017024805 A1 WO 2017024805A1 CN 2016078481 W CN2016078481 W CN 2016078481W WO 2017024805 A1 WO2017024805 A1 WO 2017024805A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
file
node
source station
target node
Prior art date
Application number
PCT/CN2016/078481
Other languages
English (en)
French (fr)
Inventor
沈坤
刘强
Original Assignee
腾讯科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技(深圳)有限公司 filed Critical 腾讯科技(深圳)有限公司
Publication of WO2017024805A1 publication Critical patent/WO2017024805A1/zh
Priority to US15/712,419 priority Critical patent/US20180013610A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1074Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
    • H04L67/1078Resource delivery mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40Support for services or applications

Definitions

  • the present invention relates to the field of communications technologies, and in particular, to a file distribution method, apparatus, and system.
  • CDN Content Distribution Network
  • HTTP Hypertext Transfer Protocol
  • rsync remote synchronization
  • rsync remote synchronization
  • the inventors of the present invention have found that in the prior art solution, if there are many target nodes, the source server will generate a large load, resulting in a decrease in the stability of the source server. Moreover, depending on the throughput of the source server, the distribution speed is also slow.
  • the embodiment of the invention provides a file distribution method, device and system, which can reduce the load of the source server, improve the stability of the source server, and improve the distribution speed.
  • An embodiment of the present invention provides a file distribution method, including:
  • a node is specified from the completion node as a source station corresponding to the task request;
  • the source server is specified as the source station corresponding to the task request;
  • An embodiment of the present invention further provides another file distribution method, including:
  • a task execution result indicating that the file is successfully obtained is reported to the task management system as a source station for obtaining files by other target nodes.
  • the embodiment of the present invention further provides a task management system, including:
  • a receiving unit configured to receive a task request for a file distribution task sent by the target node
  • a determining unit configured to determine, according to the task request, whether there is a completion node, where the completion node is another target node that has completed the file distribution task;
  • a specifying unit configured to: when the determining unit determines that the completion node exists, specify one node from the completion node as the source station corresponding to the task request, and determine, in the determining unit, that there is no completion node, and specify the source server as the task request Corresponding source station;
  • a sending unit configured to send a file distribution task to the target node according to the specified source station, so that the target node acquires a file from the specified source station.
  • an embodiment of the present invention further provides a file distribution apparatus, including:
  • a sending unit configured to send a task request for a file distribution task to the task management system
  • a receiving unit configured to receive a file distribution task returned by the task management system according to the task request
  • An obtaining unit configured to acquire a file from a source station according to the file distribution task, and store the file
  • the reporting unit is configured to report a task execution result indicating that the file acquisition succeeds to the task management system, so as to serve as a source station for obtaining files by other target nodes.
  • the embodiment of the present invention further provides a file distribution system, which includes any task management system and file distribution device provided by the embodiments of the present invention.
  • an embodiment of the present invention further provides a storage medium, where processor executable instructions are stored, and the processor executable instructions are configured to perform the following operations:
  • a node is specified from the completion node as a source station corresponding to the task request;
  • the source server is specified as the source station corresponding to the task request;
  • the embodiment of the present invention determines whether there is another target node that has completed the file distribution task (ie, determines whether there is a completion node), and if there is a completion node, the slave node The specified one of the nodes in the node is the source station corresponding to the task request. Otherwise, if there is no completion node, the source server is specified as the source station corresponding to the task request, and then the file distribution task is delivered according to the specified source station.
  • the solution can utilize the target node of the completed file distribution task as a source station for other target nodes to pull files, therefore, compared with the prior art
  • the load of the source server can be reduced, and the stability of the source server can be improved.
  • the number of target nodes that have completed the file distribution task increases, the number of source stations also increases. With the increase, the distribution speed will be greatly improved; in addition, compared to the limited source service As far as the device is concerned, the number of target nodes is large and the distribution is relatively wide. Therefore, it is more convenient to select a shorter completed node as the source station, which can further improve the distribution speed; in short, the solution can not only reduce the load of the source server, Improve the stability of the source server, and also greatly increase the speed of file distribution.
  • FIG. 1a is a schematic structural diagram of a file distribution system according to an embodiment of the present invention.
  • FIG. 1b is a flowchart of a file distribution method according to an embodiment of the present invention.
  • FIG. 2 is another flowchart of a file distribution method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a scenario of a file distribution method according to an embodiment of the present invention.
  • FIG. 3b is still another flowchart of a file distribution method according to an embodiment of the present invention.
  • 4a is a schematic structural diagram of a task management system according to an embodiment of the present invention.
  • 4b is another schematic structural diagram of a task management system according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a file distribution apparatus according to an embodiment of the present invention.
  • Embodiments of the present invention provide a file distribution method, apparatus, and system.
  • the file distribution system may include a task management system and a target node, and a client is deployed on the target node, where the file distribution device provided by the embodiment of the present invention is integrated in the client;
  • the distribution system can also include a source server that can provide files for distribution.
  • a task distribution task may be registered in the task management system, and each target node periodically sends a task request for the file distribution task to the task management system, and after receiving the task request, the task management system according to the task Requesting to determine whether there is another target node that has completed the file distribution task, that is, completing the node, if there is a completion node, designating a node from the completion node as the source station corresponding to the task request; if there is no completion node, specifying
  • the source server serves as the source station corresponding to the task request, and then delivers the file distribution task to the target node according to the specified source station, and the target node obtains the file from the specified source station, and reports the result to the task management system after the task ends.
  • the dashed line indicates the interaction of signaling and the solid line indicates the transfer of file data.
  • a task management system which may be integrated into a network device, such as a management server.
  • a file distribution method includes: receiving a task request for a file distribution task sent by a target node; determining, according to the task request, whether a completion node exists, the completion node is another target node that has completed the file distribution task; And specifying a node from the completion node as the source station corresponding to the task request; if there is no completion node, specifying the source server as the source station corresponding to the task request; and delivering the file distribution task according to the specified source station The target node so that the target node gets the file from the specified source station.
  • the specific process of the file distribution method can be as follows:
  • a file distribution task can be registered in the task management system.
  • the corresponding task information such as the information of the file to be distributed, can be obtained from the source server, and then the file distribution task is generated according to the task information, and then, the target is generated.
  • the node periodically sends a task request for the file distribution task to the task management system to obtain a file distribution task.
  • a task period that is, step 101 is specific.
  • the task request for the file distribution task sent by the target node according to the task period is received.
  • the length of the task cycle can be set according to the requirements of the actual application, and will not be described here.
  • step 102 Determine, according to the task request, whether there is a completion node. If there is a completion node, execute step 103. If there is no completion node, execute step 104.
  • the completion node is another target node that has completed the file distribution task. For example, if both target nodes A and B have successfully obtained the file indicated by the file distribution task from the source station, it indicates that the target nodes A and B have completed the file distribution task, so it can be determined that the target nodes A and B are completed. Node, and so on.
  • the completion nodes may be added to the same set, such as a complete node set, where the completed node set may be implemented in various forms, for example, the node identifiers of the respective completed nodes may be added to the completed node list. And determining whether there is a completion node by judging whether there is a node identifier of the completion node in the completion node list; or, each target node may be added to the target node list, and setting a corresponding task for each target node. Completion status, if the target node has completed the file distribution task, set its task completion status to "Complete”. Otherwise, if the target node does not complete the file distribution task, set its task completion status to "Incomplete", thus, By determining the task completion status in the target node list, it can be determined whether there is a completion node, and so on.
  • a node is specified from the completion node as a source station corresponding to the task request.
  • the attribute information and the current load information of the completion nodes may be specifically obtained, and then, according to the attribute information and the current load information, a node is selected according to the preset policy as the source station corresponding to the task request.
  • the preset policy may be set according to the requirements of the actual application. For example, the node with the lightest load may be selected as the source station corresponding to the task request, or the node with the fastest transmission speed may be selected as the source station corresponding to the task request. Alternatively, the nearest node may be selected as the source station corresponding to the task request, or the combination capability may be selected to be the strongest, for example, weights may be set for parameter distribution such as load, speed, and distance, and the values according to these parameters may be used. And the weight calculation node scores, and then selects the node with the highest score based on the score as the source station corresponding to the task request, and so on, and is not enumerated here.
  • the file distribution method may further include:
  • the step of executing the specified source server as the source station corresponding to the file distribution task request If allowed, the step of executing the specified source server as the source station corresponding to the file distribution task request;
  • the target node is instructed to wait until the next task cycle.
  • the address of the specified source station can be obtained, such as the source station's Internet Protocol (IP, Internet). Protocol), etc., then generate a file distribution task carrying the address, and send the generated file distribution task to the target node, so that the target node obtains the file from the corresponding source station according to the address.
  • IP Internet Protocol
  • Protocol Internet Protocol
  • the task execution result reported by the target node may be received. If the task execution result indicates that the file is successfully obtained, the target node is determined to be a completed node, and if If the task execution result indicates that the file acquisition fails, the number of acquisition failures of the source station is increased. For example, the number of acquisition failures of the source station may be increased by one, and so on.
  • the cause of the failure may also be detected first, for example, whether the source station has a problem, or a problem of the target node, or a link problem. , etc., if it is a problem of the source station, the number of acquisition failures of the source station is increased. Otherwise, the download may be attempted, that is, the file is re-acquired from the source station, and so on.
  • the file distribution method may further include:
  • the source station that has acquired the number of failed failures exceeds the preset threshold is deleted from the completion node.
  • the preset threshold can be set according to the requirements of the actual application, and details are not described herein again.
  • the present embodiment determines whether there are other target nodes that have completed the file distribution task (ie, whether there is a completion node), if there is a completion node Then, a node is designated as the source station corresponding to the task request from the completion node. Otherwise, if there is no completion node, the source server is specified as the source station corresponding to the task request, and then the file is distributed according to the specified source station.
  • the task is given to the target node, so that the target node obtains the file from the specified source station; since the solution can utilize the target node of the completed file distribution task as the source station for other target nodes to pull the file, therefore, relative to the current In the technology, the solution can only pull files directly from the source server, which can reduce the load of the source server and improve the stability of the source server.
  • the number of target nodes that have completed the file distribution task increases, the number of source stations increases. It will also increase its distribution speed as it increases; in addition, it is relatively limited In terms of the source server, the number of target nodes is large and the distribution is relatively wide. Therefore, it is more convenient to select a shorter completed node as the source station, which can further improve the distribution speed; in short, the solution can not only reduce the load of the source server. Improve the stability of the source server, and also greatly increase the speed of file distribution.
  • This embodiment will be described from the perspective of a file distribution device, which may be integrated in a client, and the client may serve as a target node in the embodiment of the present invention.
  • a file distribution method includes: transmitting a task request for a file distribution task to a task management system; receiving a file distribution task returned by the task management system according to the task request; acquiring a file from the source station according to the file distribution task, and storing the file Reporting the task execution result indicating that the file was successfully obtained to the task management system as a source station for the other target nodes to acquire the file.
  • the specific process of the file distribution method can be as follows:
  • a file distribution task may be registered in the task management system, and then the target node periodically sends a task request for the file distribution task to the task management system to obtain a file distribution task, that is, step 201 may specifically be as follows:
  • a task request for a file distribution task is sent to the task management system according to the task period.
  • the length of the task cycle can be set according to the requirements of the actual application, and will not be described here.
  • the receiving task management system distributes the task according to the file returned by the task request.
  • the file distribution task may carry information such as an address of the source station and file storage information.
  • the corresponding source station may be determined according to the address of the source station, the file is obtained from the determined source station, and the obtained file is stored in the location indicated by the file storage information, and the like.
  • the client A can obtain the file from the client B, and store the obtained file in the file storage information.
  • the indicated location if the source station is the source server, then at this time, client A can obtain the file from the source server, store the obtained file in the location indicated by the file storage information, and the like.
  • the task management system After the task management system receives the task execution result sent by the target node indicating that the file is successfully acquired, it can determine that the target node is a completed node as a candidate node of the source station. If the target node is selected as the source station, the file may be downloaded to the other target node, that is, after the task execution result is reported to the task management system, the file distribution method may further include:
  • the stored file is sent to the other target node.
  • the task execution system may report the task execution result indicating that the file acquisition fails, so that the task management system can perform the task execution result according to the task execution system. Further processing, for example, the task management system may increase the number of acquisition failures of the source station, such as adding one, etc., and when the number of acquisition failures exceeds a preset threshold, it may be deleted from the completion node, etc., specifically See the first embodiment, and details are not described herein again.
  • the task request for the file distribution task is sent to the task management system by the target node to obtain the file distribution task, and after the task is completed, the task execution result indicating that the file acquisition succeeds is reported to the task management system.
  • the solution can utilize the target node of the completed file distribution task as the source station for the other target nodes to pull the file, it can be reduced compared with the prior art that the file can only be directly pulled from the source server.
  • the load of the source server improves the stability of the source server; moreover, as the number of target nodes that have completed the file distribution task increases, the number of source stations increases, and the distribution speed increases greatly.
  • the number of target nodes is large and the distribution is relatively wide. Therefore, it is more convenient to select a shorter completed node as the source station, which can further improve the distribution speed; in short, the solution can not only reduce the source server.
  • the load improves the stability of the source server, and can also greatly increase the speed of file distribution.
  • the file distribution system may include a task management system and a target node, and a client is deployed on the target node, wherein any file distribution device provided by the embodiment of the present invention is integrated in the client, and
  • the file distribution system can also include a source server that can provide files for distribution.
  • the client may include client A, client B, client C, client D, etc., when client file A, client B, client C, and client are required for file distribution.
  • the terminal D sends a task request for the file distribution task to the task management system.
  • the task management system determines whether there is a completion node according to the task request, for example, when the task management system receives the client A and the client B.
  • the task management system determines that there is no completion node, and the file distribution task is respectively delivered to the client A and the client B, and the source station is indicated as the source server in the task, for example, the address of the source server can be carried, and the like.
  • the terminal A and the client B can obtain the file from the source server.
  • the client A and the client B can report the task execution result to the task management system. If the task execution result indicates that the file is successfully obtained, the task management system Determine the corresponding target nodes, such as client A and client B, as completion nodes, so that when the task management system receives other clients, such as clients When C's task requests, you can select a node from the completion node, such as client A and client B as the source station.
  • client A can be used as the source station for client C to pull files.
  • the client C can also report the task execution result to the task management system. If the task management system determines that the client C is the completion node, the client C can serve as a candidate for the source station in the subsequent task request, for example, for the client. D draws files, etc., and so on.
  • the current target node is taken as an example of the client C, and one of the processes is described in detail.
  • the process of the file distribution method may be as follows:
  • client C can send a task request for a file distribution task to the task management system in accordance with the task cycle.
  • the length of the task cycle can be set according to the requirements of the actual application, and will not be described here.
  • the task management system determines, according to the task request, whether there is a completion node. If there is a completion node, step 303 is performed. If there is no completion node, step 304 is performed.
  • step 303 can be performed at this time. Otherwise, if the client has not successfully obtained the file at this time, it is determined that there is no completion node, and then step 304 is performed.
  • the task management system specifies a node from the completion node as the source station corresponding to the task request.
  • the attribute information and the current load information of the completion nodes may be specifically obtained, and then, according to the attribute information and the current load information, a node is selected according to the preset policy as the source station corresponding to the task request.
  • the preset policy may be set according to the requirements of the actual application. For example, the node with the lightest load may be selected as the source station corresponding to the task request, or the node with the fastest transmission speed may be selected as the source station corresponding to the task request. Alternatively, the nearest node may be selected as the source station corresponding to the task request, or the combination capability may be selected to be the strongest, for example, weights may be set for parameter distribution such as load, speed, and distance, and the values according to these parameters may be used. And the weight calculation node scores, and then selects the node with the highest score based on the score as the source station corresponding to the task request, and so on, and is not enumerated here. In this embodiment, it is assumed that client A is selected to request the corresponding source station for the task.
  • the task management system determines whether the client C is allowed to directly obtain the file from the source server. If allowed, the source server may be designated as the source station; otherwise, the user is instructed to wait until the next task cycle. That is, client C can re-initiate the task request in the next task cycle.
  • the task management system sends a file distribution task to the client C according to the specified source station, where the file distribution task can carry information such as an address of the source station and file storage information.
  • the task management system may obtain the address of the specified source station, such as the IP address of the source station, and then generate a file distribution task carrying the address, and send the generated file distribution task to the client C.
  • the address of the specified source station such as the IP address of the source station
  • the IP address of the client A can be obtained, and then the file distribution task carrying the IP address of the client A is generated and sent to the client C.
  • the IP address of the source server can be obtained, and then the file distribution task carrying the IP address of the source server is generated and sent to the client C, and so on.
  • the client C acquires a file from the source station according to the received file distribution task, and stores the file.
  • client C can find client A according to the IP address of client A, obtain the file from client A, and obtain the file. Stored in the location indicated by the file storage information.
  • the client C can find the source server according to the IP address of the source server, obtain the file from the source server, and store the obtained file in the file. Store the location indicated by the information, and so on.
  • the client C reports the task execution result to the task management system.
  • the task management system receives the task execution result reported by the client C. If the task execution result indicates that the file is successfully obtained, determining that the client C is a completion node, and if the task execution result indicates that the file acquisition fails, the source station is The number of acquisition failures is increased. For example, if the source station is client A, the number of acquisition failures of client A can be increased by one, and so on.
  • the cause of the failure may also be detected first, for example, whether the source station has a problem, or a problem of the client C, or a link. The problem, etc., if the source station has a problem, the number of acquisition failures of the source station is increased. Otherwise, the download can be attempted again, that is, the file is re-acquired from the source station, and so on.
  • a threshold may be set for the number of acquisition failures, and when the number of acquisition failures exceeds a preset threshold, the corresponding source station is deleted from the completion node.
  • the preset threshold can be set according to the requirements of the actual application, and details are not described herein again.
  • the task management system used in this embodiment can determine whether there is a completion node when receiving the task request for the file distribution task sent by the target node, and if there is a completion node, specify a node from the completion node as the The source station corresponding to the task request, otherwise, if there is no completion node, the source server is specified as the source station corresponding to the task request, and then the file distribution task is delivered to the target node according to the specified source station, so that the target node is Obtain the file in the specified source station; since the scheme can use the target node of the completed file distribution task as the source station for other target nodes to pull files, it can only be pulled directly from the source server compared to the prior art.
  • the load of the source server can be reduced, and the stability of the source server can be improved.
  • the number of target nodes that have completed the file distribution task increases, the number of source stations increases, and the distribution speed thereof also increases.
  • the number of target nodes is large relative to a limited source server, The cloth is also more extensive, so it is more convenient to select a shorter completed node as the source station, which can further improve the distribution speed; in short, the solution can not only reduce the load of the source server, but also improve the stability of the source server. It can also greatly increase the speed of file distribution.
  • the average distribution time of the file can be reduced to about 19%, and the source station load can be reduced by about 80%.
  • the embodiment of the present invention further provides a task management system.
  • the task management system may include a receiving unit 401, a determining unit 402, a specifying unit 403, and a sending unit 404, as follows. :
  • the receiving unit 401 is configured to receive a task request for a file distribution task sent by the target node.
  • the receiving unit 401 may be specifically configured to receive a task request for a file distribution task sent by the target node according to a task period.
  • the length of the task cycle can be set according to the requirements of the actual application, and will not be described here.
  • the determining unit 402 is configured to determine, according to the task request, whether there is a completion node.
  • the completion node is another target node that has completed the file distribution task. For example, if both target nodes A and B have successfully obtained the file indicated by the file distribution task from the source station, it indicates that the target nodes A and B have completed the file distribution task, so it can be determined that the target nodes A and B are completed. Node, and so on.
  • the completion nodes may be added to the same set, such as a complete node set, where the completed node set may be implemented in various forms, for example, the node identifiers of the respective completed nodes may be added to the completed node list. And determining whether there is a completion node by judging whether there is a node identifier of the completion node in the completion node list; or, each target node may be added to the target node list, and setting a corresponding task for each target node. Completion status, if the target node has completed the file distribution task, set its task completion status to "Complete”. Otherwise, if the target node does not complete the file distribution task, set its task completion status to "Incomplete", thus, By determining the task completion status in the target node list, it can be determined whether there is a completion node, and so on.
  • the specifying unit 403 is configured to, when the determining unit 402 determines that there is a completion node, designate a node from the completion node as the source station corresponding to the task request, and determine, in the determining unit 402, that there is no completion node, and specify the source server as the task request. Corresponding source station.
  • the specifying unit 403 may be specifically configured to: when the determining unit 402 determines that the completion node exists, acquire the attribute information and the current load information of the completed node, and select a node according to the preset policy according to the attribute information and the current load information, as This task requests the corresponding source station.
  • the designated source server before the designated source server is used as the source station corresponding to the task request, it may also first determine whether the target node is allowed to directly obtain files from the source server, and if permitted, the source may be specified. The server acts as the source station. Otherwise, it needs to be instructed to wait until the next task period, that is, the target node can re-initiate the task request in the next task period.
  • the specifying unit 403 may be specifically configured to: when the determining unit 402 determines that the completion node is not present, if it is determined that the target node is allowed to obtain the file from the source server, specify the source server as the source station corresponding to the file distribution task request, if not Allowed, the target node is instructed to wait until the next task cycle.
  • the sending unit 404 is configured to send a file distribution task to the target node according to the specified source station, so that the target node acquires a file from the specified source station.
  • the sending unit 404 can include an obtaining subunit, a generating subunit, and a sending subunit, as follows:
  • the obtaining subunit is configured to obtain an address of the specified source station, such as an IP address of the source station.
  • the generating subunit is configured to generate a file distribution task that carries the address.
  • the sending subunit is configured to send the generated file distribution task to the target node, so that the target node acquires a file from the corresponding source station according to the address.
  • the task execution result reported by the target node may be received. If the task execution result indicates that the file is successfully obtained, the target node is determined to be a completed node, and if If the task execution result indicates that the file acquisition fails, the number of acquisition failures of the source station is increased. For example, the number of acquisition failures of the source station may be increased by one, and the like; that is, as shown in FIG. 4b, the task management system may also be configured.
  • the processing unit 405 is included as follows:
  • the receiving unit 401 is further configured to receive a task execution result reported by the target node;
  • the processing unit 405 is configured to: when the task execution result indication file acquisition succeeds, determine that the target node is a completion node; and when the task execution result indicates that the file acquisition fails, the number of acquisition failures of the source station is increased.
  • the processing unit 405 may first detect the cause of the failure, for example, whether the source station has a problem, or because the target node has a problem, or is a chain. The problem of the road, and so on, if the source station has a problem, the number of acquisition failures of the source station is increased. Otherwise, the download may be attempted, that is, the file is re-acquired from the source station, and so on.
  • the cause of the failure for example, whether the source station has a problem, or because the target node has a problem, or is a chain. The problem of the road, and so on, if the source station has a problem, the number of acquisition failures of the source station is increased. Otherwise, the download may be attempted, that is, the file is re-acquired from the source station, and so on.
  • the completion node may be considered to remove it from the candidate node of the source station (ie, the completion node), namely:
  • the processing unit 405 is further configured to delete the source station whose number of acquisition failures exceeds a preset threshold from the completion node.
  • the task management system can be specifically integrated into a network device, such as a management server.
  • the foregoing units may be implemented as the same entity, and may be implemented in any combination, and may be implemented as the same or a plurality of entities.
  • the foregoing method embodiments and details are not described herein.
  • the determining unit 402 can determine whether there are other target nodes that have completed the file distribution task (ie, determine whether the existence exists. Completion node), if there is a completion node, the designated unit 403 specifies a node from the completion node as the source station corresponding to the task request; otherwise, if there is no completion node, the specified unit 403 specifies the source server as the task request.
  • the sending unit 404 issues a file distribution task to the target node according to the specified source station, so that the target node acquires the file from the specified source station; since the solution can utilize the completed file distribution task
  • the target node acts as a source station for other target nodes to pull files. Therefore, compared with the prior art scheme that can only directly pull files from the source server, the load of the source server can be reduced, and the stability of the source server can be improved. And; as the number of target nodes that have completed the file distribution task increases, the source The number of them will also increase, and the distribution speed will be greatly improved.
  • the node can further improve the distribution speed; in short, the solution can not only reduce the load of the source server, improve the stability of the source server, but also greatly improve the file distribution speed.
  • the embodiment of the present invention further provides a file distribution apparatus.
  • the file distribution apparatus may include a sending unit 501, a receiving unit 502, an obtaining unit 503, and a reporting unit 504, as follows:
  • the sending unit 501 is configured to send a task request for a file distribution task to the task management system.
  • the sending unit 501 may specifically send a task request for a file distribution task to the task management system according to a task period.
  • the length of the task cycle can be set according to the requirements of the actual application, and will not be described here.
  • the receiving unit 502 is configured to receive a file distribution task returned by the task management system according to the task request.
  • the file distribution task may carry information such as an address of the source station and file storage information.
  • the obtaining unit 503 is configured to acquire a file from the source station according to the file distribution task, and store the file.
  • the obtaining unit 503 may be specifically configured to determine a corresponding source station according to an address of the source station, obtain a file from the determined source station, and store the obtained file in a location indicated by the file storage information.
  • the obtaining unit 503 can obtain the file from the client B, and store the obtained file in the location indicated by the file storage information; and if the source station is the source server, obtain the file.
  • Unit 503 can retrieve the file from the source server and store the retrieved file at the location indicated by the file storage information, and so on.
  • the reporting unit 504 is configured to report a task execution result indicating that the file acquisition is successful to the task management system, so as to serve as a source station for acquiring files by other target nodes.
  • the task management system After the task management system receives the task execution result sent by the target node indicating that the file is successfully acquired, it can determine that the target node is a completed node as a candidate node of the source station. If the target node is selected as the source station, the download of the file can be provided to other target nodes, namely:
  • the sending unit 501 is further configured to send the stored file to the other target node when receiving a file acquisition request of another target node.
  • the reporting unit 504 may report the task execution result indicating that the file acquisition fails to the task management system, that is,
  • the reporting unit 504 is further configured to report, to the task management system, a task execution result indicating that the file acquisition fails when the file is failed to be obtained from the source station according to the file distribution task.
  • the task management system can further process the execution result according to the task.
  • the task management system can increase the number of acquisition failures of the source station, such as adding one, and when the number of acquisition failures exceeds a preset threshold, It is deleted from the completion node, and so on.
  • the task management system can increase the number of acquisition failures of the source station, such as adding one, and when the number of acquisition failures exceeds a preset threshold, It is deleted from the completion node, and so on.
  • the file distribution device may be specifically integrated in a client, and the client may serve as a target node of an embodiment of the present invention.
  • the foregoing units may be implemented as the same entity, and may be implemented in any combination, and may be implemented as the same or a plurality of entities.
  • the foregoing method embodiments and details are not described herein.
  • the sending unit 501 of the file distributing apparatus of the present embodiment can send a task request for the file distributing task to the task management system to acquire the file distributing task, and after the obtaining unit 503 completes the task, the reporting unit 504 The task management system reports the task execution result indicating that the file is successfully acquired, so as to serve as a source station for other target nodes to obtain files; since the solution can utilize the target node of the completed file distribution task as the source station, for other target nodes to pull the file.
  • the load of the source server can be reduced, and the stability of the source server can be improved; and, as the target node of the file distribution task has been completed, As the number of source stations increases, the distribution speed will also increase greatly. In addition, the number of target nodes is large and distributed over a limited number of source servers, so routing can be more conveniently selected. A shorter completion node as the source station can further increase the distribution speed ; All in all, the program can not only reduce the load on the source server, to improve the stability of the source server, but also can greatly improve the speed of file distribution.
  • the embodiment of the present invention further provides a file distribution system, which includes any of the task management systems and any file distribution device provided by the embodiments of the present invention, wherein the task management system can be specifically referred to the fourth embodiment, file distribution.
  • the file distribution device may be specifically integrated in the client.
  • the file distribution system includes a task management system and a client.
  • a task management system configured to receive a task request for a file distribution task sent by the client; determine, according to the task request, whether there is a completion node, where the completion node is another target node that has completed the file distribution task; if the completion node is present, Specifying a node from the completion node as the source station corresponding to the task request; if there is no completion node, specifying the source server as the source station corresponding to the task request; and distributing the file distribution task to the client according to the specified source station So that the client gets the file from the specified source station.
  • a client configured to send a task request for a file distribution task to the task management system; receive a file distribution task returned by the task management system according to the task request; acquire a file from the source station according to the file distribution task, and store the file.
  • the client may also be used to report the task execution result indicating that the file acquisition succeeds to the task management system, so as to serve as a source station for other target nodes to obtain files.
  • the task management system may be further configured to receive a task execution result reported by the client, and if the task execution result indicates that the file is successfully obtained, determining that the client is a completion node, and if the task execution result indicates that the file acquisition fails.
  • the number of acquisition failures of the source station is increased, for example, the number of acquisition failures of the source station may be increased by one, and the like.
  • the client can also accept file acquisition requests from other nodes, namely:
  • the client may also be configured to send the stored file to the other target node when receiving a file acquisition request of another target node.
  • the completion node may be considered to remove it from the candidate node of the source station (ie, the completion node), namely:
  • the task management system can also be used to delete the source station whose acquisition failure exceeds the preset threshold from the completion node.
  • the preset threshold can be set according to the requirements of the actual application, and details are not described herein again.
  • the file distribution system can also include a source server as follows:
  • the source server that provides the files that need to be distributed may be sent to the task management system, and the task management system registers the file distribution task according to the information of the file to be distributed, and then the target node (ie, the client) periodically sends the task request. and many more.
  • the file distribution system can include any of the task management systems and any of the file distribution devices provided by the embodiments of the present invention, any task management system and any file provided by the embodiments of the present invention can be implemented.
  • any task management system and any file provided by the embodiments of the present invention can be implemented.
  • the program may be stored in a computer readable storage medium, and the storage medium may include: Read only memory (ROM, Read Only Memory), Random Access Memory (RAM), disk or CD.
  • ROM Read only memory
  • RAM Random Access Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种文件分发方法,包括:接收目标节点发送的关于文件分发任务的任务请求;根据该任务请求判断是否存在完成节点;若存在,则从该完成节点中指定一个节点作为该任务请求对应的源站;若不存在,则指定源服务器作为该任务请求对应的源站;根据指定的源站下发文件分发任务给该目标节点;此外,还提供相应的装置和***。

Description

一种文件分发方法、装置和***
本申请要求于2015年8月12日提交中国专利局、申请号为2015104929098、发明名称为“一种文件分发方法、装置和***”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本发明涉及通信技术领域,具体涉及一种文件分发方法、装置和***。
背景技术
内容分发网络(CDN,Content Delivery Network)是近年来较为常见的一种文件分发技术,它的基本原理是广泛采用各种缓存服务器,将这些缓存服务器分布到用户访问相对集中的地区或网络中,在用户访问网站时,利用全局负载技术将用户的访问指向距离最近的工作正常的缓存服务器上,由缓存服务器直接响应用户请求。其中,这些缓存服务器相当于源站,也称为源服务器,在对大文件进行分发时,通常是使用超文本传输协议(HTTP,HyperText Transfer Protocol)、远程同步(rsync, remote sync)、甚至是一些私有协议,然后由分发的目标节点和源服务器直接交互来获取数据的。
技术问题
在对现有技术的研究和实践过程中,本发明的发明人发现,在现有的技术方案中,如果目标节点较多,源服务器就会产生很大的负载,导致源服务器的稳定性降低,而且,受制于源服务器的吞吐量,分发速度也比较慢。
技术解决方案
本发明实施例提供一种文件分发方法、装置和***,可以降低源服务器的负载,提高源服务器的稳定性,以及提高分发速度。
本发明实施例提供一种文件分发方法,包括:
接收目标节点发送的关于文件分发任务的任务请求;
根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
若存在完成节点,则从所述完成节点中指定一个节点作为所述任务请求对应的源站;
若不存在完成节点,则指定源服务器作为所述任务请求对应的源站;
根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
本发明实施例还提供另一种文件分发方法,包括:
向任务管理***发送关于文件分发任务的任务请求;
接收任务管理***根据所述任务请求返回的文件分发任务;
根据所述文件分发任务从源站中获取文件,并存储文件;
向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
相应的,本发明实施例还提供一种任务管理***,包括:
接收单元,用于接收目标节点发送的关于文件分发任务的任务请求;
判断单元,用于根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
指定单元,用于在判断单元确定存在完成节点时,从所述完成节点中指定一个节点作为所述任务请求对应的源站,在判断单元确定不存在完成节点,指定源服务器作为所述任务请求对应的源站;
下发单元,用于根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
相应的,本发明实施例还提供一种文件分发装置,包括:
发送单元,用于向任务管理***发送关于文件分发任务的任务请求;
接收单元,用于接收任务管理***根据所述任务请求返回的文件分发任务;
获取单元,用于根据所述文件分发任务从源站中获取文件,并存储文件;
上报单元,用于向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
此外,本发明实施例还提供一种文件分发***,包括本发明实施例提供的任一种任务管理***和文件分发装置。
另外,本发明实施例还提供一种存储介质,其内存储有处理器可执行指令,所述处理器可执行指令用于执行如下操作:
接收目标节点发送的关于文件分发任务的任务请求;
根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
若存在完成节点,则从所述完成节点中指定一个节点作为所述任务请求对应的源站;
若不存在完成节点,则指定源服务器作为所述任务请求对应的源站;
根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
有益效果
本发明实施例采用在接收到目标节点发送的关于文件分发任务的任务请求时,判断是否存在已完成该文件分发任务的其他目标节点(即判断是否存在完成节点),若存在完成节点,则从完成节点中指定一个节点作为该任务请求对应的源站,否则,若不存在完成节点,则指定源服务器作为该任务请求对应的源站,然后,根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件;由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
附图说明
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1a是本发明实施例提供的文件分发***的架构示意图;
图1b是本发明实施例提供的文件分发方法的流程图;
图2是本发明实施例提供的文件分发方法的另一流程图;
图3a是本发明实施例提供的文件分发方法的场景示意图;
图3b是本发明实施例提供的文件分发方法的又一流程图;
图4a是本发明实施例提供的任务管理***的结构示意图;
图4b是本发明实施例提供的任务管理***的另一结构示意图;
图5是本发明实施例提供的文件分发装置的结构示意图。
本发明的最佳实施方式
下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。
本发明实施例提供一种文件分发方法、装置和***。
参见图1a,该文件分发***可以包括任务管理***和目标节点,目标节点上部署有客户端,其中,客户端中集成有本发明实施例所提供的任一种文件分发装置;此外,该文件分发***还可以包括源服务器,该源服务器可提供需进行分发的文件。
当需要进行文件分发时,可在任务管理***中注册一任务分发任务,并由各个目标节点定时向任务管理***发送关于文件分发任务的任务请求,任务管理***接收到任务请求后,根据该任务请求判断是否存在已完成该文件分发任务的其他目标节点,即完成节点,若存在完成节点,则从该完成节点中指定一个节点作为该任务请求对应的源站;若不存在完成节点,则指定源服务器作为该任务请求对应的源站,然后根据指定的源站下发文件分发任务给该目标节点,由该目标节点从指定的源站中获取文件,任务结束后上报结果给任务管理***,例如,可参见图1a,其中虚线表示信令的交互,实线表示文件数据的传递。
以下将分别进行详细说明。
实施例一、
在本实施例中,将从任务管理***的角度进行描述,该任务管理***具体可以集成在网络设备,如管理服务器等设备中。
一种文件分发方法,包括:接收目标节点发送的关于文件分发任务的任务请求;根据该任务请求判断是否存在完成节点,该完成节点为已完成该文件分发任务的其他目标节点;若存在完成节点,则从该完成节点中指定一个节点作为该任务请求对应的源站;若不存在完成节点,则指定源服务器作为该任务请求对应的源站;根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件。
如图1b所示,该文件分发方法的具体流程可以如下:
101、接收目标节点发送的关于文件分发任务的任务请求。
例如,可以在任务管理***中注册一个文件分发任务,比如,可以从源服务器中获取相应的任务信息,如需要进行分发的文件的信息,然后根据该任务信息生成文件分发任务,然后,由目标节点定时向任务管理***发送关于文件分发任务的任务请求,以获取文件分发任务,其中,为了描述方便,在本发明实施例中,将发送任务请求的一个周期称为任务周期,即步骤101具体可以如下:
接收目标节点按照任务周期发送的关于文件分发任务的任务请求。
其中,任务周期的长短可以根据实际应用的需求进行设置,在此不再赘述。
102、根据该任务请求判断是否存在完成节点,若存在完成节点,则执行步骤103,若不存在完成节点,则执行步骤104。
其中,该完成节点为已完成该文件分发任务的其他目标节点。比如,如果目标节点A和B均已从源站成功获取了该文件分发任务所指示的文件,则表明目标节点A和B已完成该文件分发任务,所以,可以确定目标节点A和B为完成节点,等等。
具体实施时,可以将完成节点都添加至同一集合,如完成节点集合中,其中,该完成节点集合可以通过多种形式来实现,比如,可以将各个完成节点的节点标识添加至完成节点列表中,然后,通过判断该完成节点列表中是否存在完成节点的节点标识来确定是否存在完成节点;或者,也可以将各个目标节点都添加至目标节点列表中,并为每一个目标节点设置相应的任务完成状态,若目标节点已完成文件分发任务,则将其任务完成状态设置为“完成”,否则,若目标节点未完成文件分发任务,则将其任务完成状态设置为“未完成”,这样,通过判断该目标节点列表中的任务完成状态便可以确定是否存在完成节点,等等。
103、在确定存在完成节点时,从该完成节点中指定一个节点作为该任务请求对应的源站。
例如,具体可以获取这些完成节点的属性信息和当前负载信息,然后根据该属性信息和当前负载信息,按照预置策略选择一个节点,作为该任务请求对应的源站。
其中,预置策略可以根据实际应用的需求进行设置,比如,可以选择负载最轻的节点作为该任务请求对应的源站,也可以选择传送速度最快的节点作为该任务请求对应的源站,或者,还可以选择距离最近的节点作为该任务请求对应的源站,或者,还可以选择综合能力最强的,比如,可以为负载、速度和距离等参数分布设置权重,并根据这些参数的值和权重计算节点的评分,然后基于评分选择分值最高的节点作为该任务请求对应的源站,等等,在此不再列举。
104、在确定不存在完成节点时,指定源服务器作为该任务请求对应的源站。
可选的,为了减轻源服务器的负载,在将指定源服务器作为该任务请求对应的源站之前,还可以先确定是否允许该目标节点直接从源服务器中获取文件,若允许,才可指定源服务器作为源站,否则,需指示其等待至下一任务周期,即目标节点可在下一任务周期,再重新发起任务请求。也就是说,在步骤“指定源服务器作为该文件分发任务请求对应的源站”之前,该文件分发方法还可以包括:
判断是否允许该目标节点从源服务器中获取文件;
若允许,则执行指定源服务器作为该文件分发任务请求对应的源站的步骤;
若不允许,则指示该目标节点等待至下一任务周期。
105、根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件。
例如,具体可以获取指定的源站的地址,比如源站的网际协议(IP,Internet Protocol)地址等,然后生成携带该地址的文件分发任务,向该目标节点发送生成的文件分发任务,以便该目标节点根据该地址从相应的源站中获取文件。
在根据指定的源站下发文件分发任务给该目标节点之后,还可以接收该目标节点上报的任务执行结果,若该任务执行结果指示文件获取成功,则确定该目标节点为完成节点,若该任务执行结果指示文件获取失败,则对该源站的获取失败次数进行增加,比如,可以将该源站的获取失败次数加一,等等。
可选的,在对该源站的获取失败次数进行增加之前,还可以先检测失败的原因,比如,可以检测是否该源站有问题,还是因为目标节点的问题,或者,是链路的问题,等等,若是源站的问题,才对该源站的获取失败次数进行增加,否则,可以尝试重新下载,即重新从该源站中获取文件,等等。
可选的,如果多个节点以某一完成节点作为源站并且下载失败,则可以考虑将其从源站的候选节点(即完成节点)中删除,即该文件分发方法还可以包括:
将获取失败次数超过预置阈值的源站从完成节点中删除。
其中,该预置阈值可以根据实际应用的需求进行设置,在此不再赘述。
由上可知,本实施例采用在接收到目标节点发送的关于文件分发任务的任务请求时,判断是否存在已完成该文件分发任务的其他目标节点(即判断是否存在完成节点),若存在完成节点,则从完成节点中指定一个节点作为该任务请求对应的源站,否则,若不存在完成节点,则指定源服务器作为该任务请求对应的源站,然后,根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件;由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
实施例二、
本实施例将从文件分发装置的角度进行描述,该文件分发装置具体可以集成在客户端中,该客户端可作为本发明实施例的目标节点。
一种文件分发方法,包括:向任务管理***发送关于文件分发任务的任务请求;接收任务管理***根据该任务请求返回的文件分发任务;根据该文件分发任务从源站中获取文件,并存储文件;向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
如图2所示,该文件分发方法的具体流程可以如下:
201、向任务管理***发送关于文件分发任务的任务请求。
例如,可以在任务管理***中注册一个文件分发任务,然后,由目标节点定时向任务管理***发送关于文件分发任务的任务请求,以获取文件分发任务即步骤201具体可以如下:
按照任务周期向任务管理***发送关于文件分发任务的任务请求。
其中,任务周期的长短可以根据实际应用的需求进行设置,在此不再赘述。
202、接收任务管理***根据该任务请求返回的文件分发任务。
其中,该文件分发任务中可以携带源站的地址和文件存储信息等信息。
此外,任务管理***根据该任务请求返回文件分发任务的具体操作可参见实施例一,在此不再赘述。
203、根据该文件分发任务从源站中获取文件,并存储文件;
例如,具体可以根据源站的地址确定相应的源站,从确定的源站中获取文件,并将获取到的文件存储在该文件存储信息所指示的位置,等等。
比如,以该目标节点为客户端A为例,若源站为客户端B,则此时,客户端A可以从客户端B中获取文件,并将获取到的文件存储在该文件存储信息所指示的位置;而若源站为源服务器,则此时,客户端A可以从源服务器中获取文件,并将获取到的文件存储在该文件存储信息所指示的位置,等等。
204、向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
当任务管理***接收到目标节点发送的指示文件获取成功的任务执行结果后,便可以确定该目标节点为完成节点,以作为源站的候选节点。若该目标节点被选为源站,则可以向其他目标节点提供文件的下载,即可选的,在向任务管理***上报指示文件获取成功的任务执行结果之后,该文件分发方法还可以包括:
在接收到其他目标节点的文件获取请求时,将存储的文件发送给该其他目标节点。
此外,需说明的是,若根据该文件分发任务从源站中获取文件失败,则可以向任务管理***上报指示文件获取失败的任务执行结果,这样,任务管理***便可以根据该任务执行结果作进一步处理,比如,任务管理***可以对该源站的获取失败次数进行增加,比如加一等,而当获取失败次数超过预置阈值,则可以将其从完成节点中删除,等等,具体可参见实施例一,在此不再赘述。
由上可知,本实施例采用由目标节点向任务管理***发送关于文件分发任务的任务请求,以获取文件分发任务,并在完成该任务之后,向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
实施例三、
根据实施例一和二所描述的方法,以下将举例作进一步详细说明。
例如,参见图3a,该文件分发***可以包括任务管理***和目标节点,目标节点上部署有客户端,其中,客户端中集成有本发明实施例所提供的任一种文件分发装置,此外,该文件分发***还可以包括源服务器,该源服务器可提供需进行分发的文件。
比如,如图3a所示,该客户端可包括客户端A、客户端B、客户端C和客户端D等,当需要进行文件分发时,客户端A、客户端B、客户端C和客户端D等向任务管理***发送关于文件分发任务的任务请求,任务管理***接收到任务请求后,根据该任务请求判断是否存在完成节点,比如,当任务管理***接收到客户端A和客户端B的任务请求时,确定不存在完成节点,于是分别下发文件分发任务给客户端A和客户端B,并在任务中指示源站为源服务器,比如可携带源服务器的地址等,这样,客户端A和客户端B便可从源服务器中获取文件,此后,客户端A和客户端B还可以将任务执行结果上报给任务管理***,若该任务执行结果指示文件获取成功,则任务管理***将相应的目标节点,如客户端A和客户端B确定为完成节点,这样,当任务管理***接收到其他客户端,如客户端C的任务请求时,便可从完成节点,如客户端A和客户端B中选择一个节点作为源站,比如,可以将客户端A作为源站,以供客户端C拉取文件,同理,客户端C也可以将任务执行结果上报给任务管理***,若任务管理***确定客户端C为完成节点,则客户端C在后续的任务请求中,可作为源站的候选,比如,供客户端D拉取文件,等等,以此类推。
基于上述场景,以下将以当前目标节点为客户端C为例,对其中的一个流程进行详细说明,如图3b所示,该文件分发方法的流程具体可以如下:
301、客户端C向任务管理***发送的关于文件分发任务的任务请求。
例如,客户端C可以按照任务周期向任务管理***发送关于文件分发任务的任务请求。
其中,任务周期的长短可以根据实际应用的需求进行设置,在此不再赘述。
302、任务管理***根据该任务请求判断是否存在完成节点,若存在完成节点,则执行步骤303,若不存在完成节点,则执行步骤304。
例如,以图3a所示的场景为例,由于此时客户端A和客户端B已均已从源站成功获取了该文件分发任务所指示的文件,因此,可以确定客户端A和客户端B为完成节点,因此,此时可以执行步骤303,否则,若此时尚未有客户端成功获取到文件,则确定不存在完成节点,于是执行步骤304。
303、在确定存在完成节点时,任务管理***从该完成节点中指定一个节点作为该任务请求对应的源站。
例如,具体可以获取这些完成节点的属性信息和当前负载信息,然后根据该属性信息和当前负载信息,按照预置策略选择一个节点,作为该任务请求对应的源站。
其中,预置策略可以根据实际应用的需求进行设置,比如,可以选择负载最轻的节点作为该任务请求对应的源站,也可以选择传送速度最快的节点作为该任务请求对应的源站,或者,还可以选择距离最近的节点作为该任务请求对应的源站,或者,还可以选择综合能力最强的,比如,可以为负载、速度和距离等参数分布设置权重,并根据这些参数的值和权重计算节点的评分,然后基于评分选择分值最高的节点作为该任务请求对应的源站,等等,在此不再列举。在本实施例中,假设选择客户端A为该任务请求对应的源站。
304、在确定不存在完成节点时,任务管理***确定是否允许客户端C直接从源服务器中获取文件,若允许,才可指定源服务器作为源站,否则,指示其等待至下一任务周期,即客户端C可在下一任务周期,再重新发起任务请求。
305、任务管理***根据指定的源站下发文件分发任务给客户端C,其中,该文件分发任务可以携带源站的地址和文件存储信息等信息。
例如,任务管理***可以获取指定的源站的地址,比如源站的IP地址,然后生成携带该地址的文件分发任务,向客户端C发送生成的文件分发任务。
比如,若确定源站为客户端A,则此时可以获取客户端A的IP地址,然后生成携带该客户端A的IP地址的文件分发任务,并下发给客户端C。
又比如,若确定源站为源服务器,则此时可以获取源服务器的IP地址,然后生成携带该源服务器的IP地址的文件分发任务,并下发给客户端C,等等。
306、客户端C根据接收到的文件分发任务从源站中获取文件,并存储文件。
例如,如果收到的文件分发任务中指定源站为客户端A,则客户端C可以根据客户端A的IP地址查找到客户端A,从客户端A中获取文件,并将获取到的文件存储在文件存储信息所指示的位置。
又例如,如果收到的文件分发任务中指定源站为源服务器,则客户端C可以根据源服务器的IP地址查找到源服务器,从源服务器中获取文件,并将获取到的文件存储在文件存储信息所指示的位置,等等。
307、客户端C向任务管理***上报任务执行结果。
308、任务管理***接收客户端C上报的任务执行结果,若该任务执行结果指示文件获取成功,则确定该客户端C为完成节点,若该任务执行结果指示文件获取失败,则对该源站的获取失败次数进行增加,比如,若源站为客户端A,则可以将客户端A的获取失败次数加一,等等。
可选的,在对该源站的获取失败次数进行增加之前,还可以先检测失败的原因,比如,可以检测是否该源站有问题,还是因为客户端C的问题,或者,是链路的问题,等等,若是源站的问题,才对该源站的获取失败次数进行增加,否则,可以尝试重新下载,即重新从该源站中获取文件,等等。
可选的,如果多个节点以某一完成节点作为源站并且下载失败,则可以考虑将其从源站的候选节点(即完成节点)中删除,比如,若多个节点以客户端A为源站,且下载失败,则可以考虑将客户端A从完成节点中删除,比如将客户端A的文件下载状态从“完成”更改为“未完成”,让客户端A重新获取文件,以保持文件的正确性。也就是说,可以为获取失败次数设置一个阈值,在获取失败次数超过预置阈值时,将相应源站从完成节点中删除。其中,该预置阈值可以根据实际应用的需求进行设置,在此不再赘述。
应当理解的是,其他客户端的执行流程与此类似,在此不再赘述。
由上可知,本实施例采用的任务管理***在接收到目标节点发送的关于文件分发任务的任务请求时,可以判断是否存在完成节点,若存在完成节点,则从完成节点中指定一个节点作为该任务请求对应的源站,否则,若不存在完成节点,则指定源服务器作为该任务请求对应的源站,然后,根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件;由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
需说明的是,将本方案应用到实际的生产环境后,文件的平均分发用时可降低到了原先的19%左右,而源站负载可降低80%左右。
实施例四、
为了更好地实施以上方法,本发明实施例还提供一种任务管理***,如图4a所示,该任务管理***可以包括接收单元401、判断单元402、指定单元403和下发单元404,如下:
(1)接收单元401;
接收单元401,用于接收目标节点发送的关于文件分发任务的任务请求。
例如,接收单元401,具体可以用于接收目标节点按照任务周期发送的关于文件分发任务的任务请求。
其中,任务周期的长短可以根据实际应用的需求进行设置,在此不再赘述。
(2)判断单元402;
判断单元402,用于根据该任务请求判断是否存在完成节点。
其中,该完成节点为已完成该文件分发任务的其他目标节点。比如,如果目标节点A和B均已从源站成功获取了该文件分发任务所指示的文件,则表明目标节点A和B已完成该文件分发任务,所以,可以确定目标节点A和B为完成节点,等等。
具体实施时,可以将完成节点都添加至同一集合,如完成节点集合中,其中,该完成节点集合可以通过多种形式来实现,比如,可以将各个完成节点的节点标识添加至完成节点列表中,然后,通过判断该完成节点列表中是否存在完成节点的节点标识来确定是否存在完成节点;或者,也可以将各个目标节点都添加至目标节点列表中,并为每一个目标节点设置相应的任务完成状态,若目标节点已完成文件分发任务,则将其任务完成状态设置为“完成”,否则,若目标节点未完成文件分发任务,则将其任务完成状态设置为“未完成”,这样,通过判断该目标节点列表中的任务完成状态便可以确定是否存在完成节点,等等。
(3)指定单元403;
指定单元403,用于在判断单元402确定存在完成节点时,从该完成节点中指定一个节点作为该任务请求对应的源站,在判断单元402确定不存在完成节点,指定源服务器作为该任务请求对应的源站。
例如,指定单元403,具体可以用于在判断单元402确定存在完成节点时,获取该完成节点的属性信息和当前负载信息,根据该属性信息和当前负载信息,按照预置策略选择一个节点,作为该任务请求对应的源站。
可选的,为了减轻源服务器的负载,在将指定源服务器作为该任务请求对应的源站之前,还可以先确定是否允许该目标节点直接从源服务器中获取文件,若允许,才可指定源服务器作为源站,否则,需指示其等待至下一任务周期,即目标节点可在下一任务周期,再重新发起任务请求。即:
该指定单元403,具体可以用于在判断单元402确定不存在完成节点时,若确定允许该目标节点从源服务器中获取文件,则指定源服务器作为该文件分发任务请求对应的源站,若不允许,则指示该目标节点等待至下一任务周期。
(4)下发单元404;
下发单元404,用于根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件。
例如,该下发单元404可以包括获取子单元、生成子单元和发送子单元,如下:
该获取子单元,用于获取指定的源站的地址,比如源站的IP地址等。
该生成子单元,用于生成携带该地址的文件分发任务。
该发送子单元,用于向该目标节点发送生成的文件分发任务,以便该目标节点根据该地址从相应的源站中获取文件。
在根据指定的源站下发文件分发任务给该目标节点之后,还可以接收该目标节点上报的任务执行结果,若该任务执行结果指示文件获取成功,则确定该目标节点为完成节点,若该任务执行结果指示文件获取失败,则对该源站的获取失败次数进行增加,比如,可以将该源站的获取失败次数加一,等等;即如图4b所示,该任务管理***还可以包括处理单元405,如下:
接收单元401,还可以用于接收该目标节点上报的任务执行结果;
处理单元405,可以用于在该任务执行结果指示文件获取成功时,确定该目标节点为完成节点;在该任务执行结果指示文件获取失败时,对该源站的获取失败次数进行增加。
可选的,在对该源站的获取失败次数进行增加之前,处理单元405还可以先检测失败的原因,比如,可以检测是否该源站有问题,还是因为目标节点的问题,或者,是链路的问题,等等,若是源站的问题,才对该源站的获取失败次数进行增加,否则,可以尝试重新下载,即重新从该源站中获取文件,等等。
可选的,如果多个节点以某一完成节点作为源站并且下载失败,则可以考虑将其从源站的候选节点(即完成节点)中删除,即:
该处理单元405,还可以用于将获取失败次数超过预置阈值的源站从完成节点中删除。
该任务管理***具体可以集成在网络设备,如管理服务器等设备中。
具体实施时,以上各个单元可以作为同一实体实现,也可以进行任意组合,作为同一或若干个实体来实现,以上各个单元的具体实施可参见前面的方法实施例,在此不再赘述。
由上可知,本实施例的任务管理***在接收到目标节点发送的关于文件分发任务的任务请求时,可以由判断单元402判断是否存在已完成该文件分发任务的其他目标节点(即判断是否存在完成节点),若存在完成节点,则由指定单元403从完成节点中指定一个节点作为该任务请求对应的源站,否则,若不存在完成节点,则由指定单元403指定源服务器作为该任务请求对应的源站,然后,由下发单元404根据指定的源站下发文件分发任务给该目标节点,以便该目标节点从指定的源站中获取文件;由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
实施例五、
相应的,本发明实施例还提供一种文件分发装置,如图5所示,该文件分发装置可以包括发送单元501、接收单元502、获取单元503和上报单元504,如下:
发送单元501,用于向任务管理***发送关于文件分发任务的任务请求。
例如,发送单元501具体可以按照任务周期向任务管理***发送关于文件分发任务的任务请求。
其中,任务周期的长短可以根据实际应用的需求进行设置,在此不再赘述。
接收单元502,用于接收任务管理***根据该任务请求返回的文件分发任务。
其中,该文件分发任务中可以携带源站的地址和文件存储信息等信息。
获取单元503,用于根据该文件分发任务从源站中获取文件,并存储文件。
例如,获取单元503,具体可以用于根据源站的地址确定相应的源站,从确定的源站中获取文件,将获取到的文件存储在该文件存储信息所指示的位置。
比如,若源站为客户端B,则获取单元503可以从客户端B中获取文件,并将获取到的文件存储在该文件存储信息所指示的位置;而若源站为源服务器,则获取单元503可以从源服务器中获取文件,并将获取到的文件存储在该文件存储信息所指示的位置,等等。
上报单元504,用于向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
当任务管理***接收到目标节点发送的指示文件获取成功的任务执行结果后,便可以确定该目标节点为完成节点,以作为源站的候选节点。若该目标节点被选为源站,则可以向其他目标节点提供文件的下载,即:
该发送单元501,还可以用于在接收到其他目标节点的文件获取请求时,将存储的文件发送给该其他目标节点。
此外,需说明的是,若接收单元502根据该文件分发任务从源站中获取文件失败,则上报单元504可以向任务管理***上报指示文件获取失败的任务执行结果,即:
上报单元504,还用于在根据该文件分发任务从源站中获取文件失败时,向任务管理***上报指示文件获取失败的任务执行结果。
这样,任务管理***便可以根据该任务执行结果作进一步处理,比如,任务管理***可以对该源站的获取失败次数进行增加,比如加一等,而当获取失败次数超过预置阈值,则可以将其从完成节点中删除,等等,具体可参见前面的实施例,在此不再赘述。
该文件分发装置具体可以集成在客户端中,该客户端可作为本发明实施例的目标节点。
具体实施时,以上各个单元可以作为同一实体实现,也可以进行任意组合,作为同一或若干个实体来实现,以上各个单元的具体实施可参见前面的方法实施例,在此不再赘述。
由上可知,本实施例的文件分发装置的发送单元501可以向任务管理***发送关于文件分发任务的任务请求,以获取文件分发任务,并在获取单元503完成该任务之后,由上报单元504向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站;由于该方案可以利用已完成文件分发任务的目标节点作为源站,以供其他目标节点拉取文件,因此,相对于现有技术中只能从源服务器直接拉取文件的方案而言,可以降低源服务器的负载,提高源服务器的稳定性;而且,随着已完成文件分发任务的目标节点的增多,源站的数量也会随着增加,其分发速度也会大大提高;此外,相对于有限的源服务器而言,目标节点数量众多,分布也较为广泛,所以,可以更方便地从中选择路由较短的完成节点作为源站,可以进一步提高分发速度;总而言之,该方案不仅可以降低源服务器的负载,提高源服务器的稳定性,而且,还可以大大提高文件的分发速度。
实施例六、
此外,本发明实施例还提供一种文件分发***,包括本发明实施例所提供的任一种任务管理***和任一种文件分发装置,其中,任务管理***具体可参见实施例四,文件分发装置具体可参见实施例五,其中,文件分发装置具体可以集成在客户端中,例如,以该文件分发***包括任务管理***和客户端为例,则具体可以如下:
任务管理***,用于接收客户端发送的关于文件分发任务的任务请求;根据该任务请求判断是否存在完成节点,该完成节点为已完成该文件分发任务的其他目标节点;若存在完成节点,则从该完成节点中指定一个节点作为该任务请求对应的源站;若不存在完成节点,则指定源服务器作为该任务请求对应的源站;根据指定的源站下发文件分发任务给该客户端,以便该客户端从指定的源站中获取文件。
客户端,用于向任务管理***发送关于文件分发任务的任务请求;接收任务管理***根据该任务请求返回的文件分发任务;根据该文件分发任务从源站中获取文件,并存储文件。
该客户端,还可以用于向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
则此时,任务管理***,还可以用于接收该客户端上报的任务执行结果,若该任务执行结果指示文件获取成功,则确定该客户端为完成节点,若该任务执行结果指示文件获取失败,则对该源站的获取失败次数进行增加,比如,可以将该源站的获取失败次数加一,等等。
若该客户端被指定为源站,则还可以接受其他节点的文件获取请求,即:
该客户端,还可以用于在接收到其他目标节点的文件获取请求时,将存储的文件发送给该其他目标节点。
可选的,如果多个节点以某一完成节点作为源站并且下载失败,则可以考虑将其从源站的候选节点(即完成节点)中删除,即:
任务管理***,还可以用于将获取失败次数超过预置阈值的源站从完成节点中删除。
其中,该预置阈值可以根据实际应用的需求进行设置,在此不再赘述。
此外,该文件分发***还可以包括源服务器,如下:
源服务器,用于提供需要进行分发的文件。比如,可以将需要进行分发的文件的信息发送给任务管理***,由任务管理***根据该需要进行分发的文件的信息注册文件分发任务,然后由各目标节点(即客户端)定时发送任务请求,等等。
以上各个设备的具体实施可参见前面的实施例,在此不再赘述。
由于该文件分发***可以包括本发明实施例所提供的任一种任务管理***和任一种文件分发装置,因此,可以实现本发明实施例所提供的任一种任务管理***和任一种文件分发装置所能实现的有益效果,具体可参见前面的实施例,在此不再赘述。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序可以存储于一计算机可读存储介质中,存储介质可以包括:只读存储器(ROM,Read Only Memory)、随机存取记忆体(RAM,Random Access Memory)、磁盘或光盘等。
以上对本发明实施例所提供的一种文件分发方法、装置和***进行了详细介绍,本文中应用了具体个例对本发明的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本发明的方法及其核心思想;同时,对于本领域的技术人员,依据本发明的思想,在具体实施方式及应用范围上均会有改变之处,综上所述,本说明书内容不应理解为对本发明的限制。

Claims (22)

  1. 一种文件分发方法,其特征在于,包括:
    接收目标节点发送的关于文件分发任务的任务请求;
    根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
    若存在完成节点,则从所述完成节点中指定一个节点作为所述任务请求对应的源站;
    若不存在完成节点,则指定源服务器作为所述任务请求对应的源站;
    根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
  2. 根据权利要求1所述的方法,其特征在于,所述根据源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件,包括:
    获取指定的源站的地址;
    生成携带所述地址的文件分发任务;
    向所述目标节点发送生成的文件分发任务,以便所述目标节点根据所述地址从相应的源站中获取文件。
  3. 根据权利要求1所述的方法,其特征在于,所述将指定源服务器作为所述任务请求对应的源站之前,还包括:
    判断是否允许所述目标节点从源服务器中获取文件;
    若允许,则执行指定源服务器作为所述文件分发任务请求对应的源站的步骤;
    若不允许,则指示所述目标节点等待至下一任务周期。
  4. 根据权利要求1至3任一项所述的方法,其特征在于,所述根据指定的源站下发文件分发任务给所述目标节点之后,还包括:
    接收所述目标节点上报的任务执行结果;
    若所述任务执行结果指示文件获取成功,则确定所述目标节点为完成节点;
    若所述任务执行结果指示文件获取失败,则对所述源站的获取失败次数进行增加。
  5. 根据权利要求4所述的方法,其特征在于,还包括:
    将获取失败次数超过预置阈值的源站从完成节点中删除。
  6. 根据权利要求1至3任一项所述的方法,其特征在于,所述从所述完成节点中指定一个节点作为所述任务请求对应的源站,包括:
    获取所述完成节点的属性信息和当前负载信息;
    根据所述属性信息和当前负载信息,按照预置策略选择一个节点,作为所述任务请求对应的源站。
  7. 一种文件分发方法,其特征在于,包括:
    向任务管理***发送关于文件分发任务的任务请求;
    接收任务管理***根据所述任务请求返回的文件分发任务;
    根据所述文件分发任务从源站中获取文件,并存储文件;
    向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
  8. 根据权利要求7所述的方法,其特征在于,所述向任务管理***上报指示文件获取成功的任务执行结果之后,还包括:
    在接收到其他目标节点的文件获取请求时,将存储的文件发送给所述其他目标节点。
  9. 根据权利要求7所述的方法,其特征在于,所述文件分发任务携带源站的地址和文件存储信息,则所述根据所述文件分发任务从源站中获取文件,并存储文件,包括:
    根据源站的地址确定相应的源站;
    从确定的源站中获取文件;
    将获取到的文件存储在所述文件存储信息所指示的位置。
  10. 根据权利要求7至9任一项所述的方法,其特征在于,所述接收任务管理***根据所述任务请求返回的文件分发任务之后,还包括:
    若根据所述文件分发任务从源站中获取文件失败,则向任务管理***上报指示文件获取失败的任务执行结果。
  11. 一种任务管理***,其特征在于,包括:
    接收单元,用于接收目标节点发送的关于文件分发任务的任务请求;
    判断单元,用于根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
    指定单元,用于在判断单元确定存在完成节点时,从所述完成节点中指定一个节点作为所述任务请求对应的源站,在判断单元确定不存在完成节点,指定源服务器作为所述任务请求对应的源站;
    下发单元,用于根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
  12. 根据权利要求11所述的***,其特征在于,所述下发单元包括获取子单元、生成子单元和发送子单元;
    所述获取子单元,用于获取指定的源站的地址;
    所述生成子单元,用于生成携带所述地址的文件分发任务;
    所述发送子单元,用于向所述目标节点发送生成的文件分发任务,以便所述目标节点根据所述地址从相应的源站中获取文件。
  13. 根据权利要求11所述的***,其特征在于,
    所述指定单元,具体用于在判断单元确定不存在完成节点时,若确定允许所述目标节点从源服务器中获取文件,则指定源服务器作为所述文件分发任务请求对应的源站,若不允许,则指示所述目标节点等待至下一任务周期。
  14. 根据权利要求11至13任一项所述的***,其特征在于,还包括处理单元;
    所述接收单元,还用于接收所述目标节点上报的任务执行结果;
    所述处理单元,用于在所述任务执行结果指示文件获取成功时,确定所述目标节点为完成节点;在所述任务执行结果指示文件获取失败时,对所述源站的获取失败次数进行增。
  15. 根据权利要求14所述的***,其特征在于,
    所述处理单元,还用于将获取失败次数超过预置阈值的源站从完成节点中删除。
  16. 根据权利要求11至13任一项所述的***,其特征在于,
    所述指定单元,具体用于在判断单元确定存在完成节点时,获取所述完成节点的属性信息和当前负载信息,根据所述属性信息和当前负载信息,按照预置策略选择一个节点,作为所述任务请求对应的源站。
  17. 一种文件分发装置,其特征在于,包括:
    发送单元,用于向任务管理***发送关于文件分发任务的任务请求;
    接收单元,用于接收任务管理***根据所述任务请求返回的文件分发任务;
    获取单元,用于根据所述文件分发任务从源站中获取文件,并存储文件;
    上报单元,用于向任务管理***上报指示文件获取成功的任务执行结果,以便作为供其他目标节点获取文件的源站。
  18. 根据权利要求17所述的装置,其特征在于,
    所述发送单元,还用于在接收到其他目标节点的文件获取请求时,将存储的文件发送给所述其他目标节点。
  19. 根据权利要求17所述的装置,其特征在于,所述文件分发任务携带源站的地址和文件存储信息,则:
    所述获取单元,具体用于根据源站的地址确定相应的源站,从确定的源站中获取文件,将获取到的文件存储在所述文件存储信息所指示的位置。
  20. 根据权利要求17至19任一项所述的装置,其特征在于,
    所述上报单元,还用于在根据所述文件分发任务从源站中获取文件失败时,向任务管理***上报指示文件获取失败的任务执行结果。
  21. 一种文件分发***,其特征在于,包括权利要求11至16任一项所述的任务管理***和权利要求17至20任一项所述的文件分发装置。
  22. 一种存储介质,其特征在于,其内存储有处理器可执行指令,所述处理器可执行指令用于执行如下操作:
    接收目标节点发送的关于文件分发任务的任务请求;
    根据所述任务请求判断是否存在完成节点,所述完成节点为已完成所述文件分发任务的其他目标节点;
    若存在完成节点,则从所述完成节点中指定一个节点作为所述任务请求对应的源站;
    若不存在完成节点,则指定源服务器作为所述任务请求对应的源站;
    根据指定的源站下发文件分发任务给所述目标节点,以便所述目标节点从指定的源站中获取文件。
PCT/CN2016/078481 2015-08-12 2016-04-05 一种文件分发方法、装置和*** WO2017024805A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/712,419 US20180013610A1 (en) 2015-08-12 2017-09-22 File delivery method, apparatus and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510492909.8 2015-08-12
CN201510492909.8A CN106453460B (zh) 2015-08-12 2015-08-12 一种文件分发方法、装置和***

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/712,419 Continuation US20180013610A1 (en) 2015-08-12 2017-09-22 File delivery method, apparatus and system

Publications (1)

Publication Number Publication Date
WO2017024805A1 true WO2017024805A1 (zh) 2017-02-16

Family

ID=57983460

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/078481 WO2017024805A1 (zh) 2015-08-12 2016-04-05 一种文件分发方法、装置和***

Country Status (3)

Country Link
US (1) US20180013610A1 (zh)
CN (1) CN106453460B (zh)
WO (1) WO2017024805A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108965362B (zh) * 2017-05-19 2023-03-24 腾讯科技(深圳)有限公司 一种数据处理方法、服务器及存储介质
CN108573071A (zh) * 2018-05-09 2018-09-25 广东工业大学 一种文件操作方法及相关装置
CN109981780A (zh) * 2019-03-27 2019-07-05 深圳市网心科技有限公司 一种调度方法、节点设备、调度***及存储介质
CN110765092A (zh) * 2019-09-20 2020-02-07 北京三快在线科技有限公司 分布式搜索***、索引分发方法及存储介质
CN110768914B (zh) * 2019-10-23 2022-03-29 上海交通大学 基于语义场景实例迁移的去中心化物联网关***
CN113676757A (zh) * 2020-05-15 2021-11-19 杭州晨熹多媒体科技有限公司 一种数据传输方法、装置和***
CN115102946B (zh) * 2022-06-16 2023-10-24 平安银行股份有限公司 一种基于文件传输的配置方法及***

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102045400A (zh) * 2010-12-29 2011-05-04 北京世纪互联工程技术服务有限公司 一种文件内容同步方法和***
CN103546546A (zh) * 2013-09-30 2014-01-29 蓝盾信息安全技术股份有限公司 一种大规模集群分发文件的方法
CN103634346A (zh) * 2012-08-23 2014-03-12 腾讯科技(深圳)有限公司 网络数据中心的数据分发方法和***
US20140140189A1 (en) * 2002-05-14 2014-05-22 Genghiscomm Holdings, LLC Cooperative Subspace Demultiplexing in Communication Networks
CN104317716A (zh) * 2014-10-30 2015-01-28 华为技术有限公司 分布式节点间的数据传输方法及分布式节点设备

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020049760A1 (en) * 2000-06-16 2002-04-25 Flycode, Inc. Technique for accessing information in a peer-to-peer network
AU2002232464A1 (en) * 2001-02-09 2002-08-28 Microsoft Corporation Distribution of binary executables and content from peer locations/machines
US7509372B2 (en) * 2001-09-13 2009-03-24 International Business Machines Corporation Method and system for redirecting data requests in peer-to-peer data networks
US20060212542A1 (en) * 2005-03-15 2006-09-21 1000 Oaks Hu Lian Technology Development Co., Ltd. Method and computer-readable medium for file downloading in a peer-to-peer network
US20060212584A1 (en) * 2005-03-15 2006-09-21 Qian Xiang Shi Ji (Beijing) Technology Development Co. Ltd. Method and system for accelerating downloading of web page content by a peer-to-peer network
FR2905488B1 (fr) * 2006-09-04 2011-04-01 Baracoda Architecture d'acces a un flux de donnees au moyen d'un terminal utilisateur
US8090813B2 (en) * 2006-09-19 2012-01-03 Solid State Networks, Inc. Methods and apparatus for data transfer
US8169916B1 (en) * 2007-11-23 2012-05-01 Media Melon, Inc. Multi-platform video delivery configuration
US8443106B2 (en) * 2007-12-21 2013-05-14 Gary Stephen Shuster Content restriction compliance using reverse DNS lookup
KR101218828B1 (ko) * 2009-07-02 2013-01-04 (주)에임투지 요청배정장치를 이용한 상호협력캐시 방법 및 컨텐츠 제공 방법
US8560604B2 (en) * 2009-10-08 2013-10-15 Hola Networks Ltd. System and method for providing faster and more efficient data communication
CN102129434B (zh) * 2010-01-13 2015-06-10 腾讯科技(北京)有限公司 读写分离数据库的方法及***
US8700945B1 (en) * 2011-09-23 2014-04-15 Amazon Technologies, Inc. Content delivery failover
US9059862B2 (en) * 2012-03-13 2015-06-16 Verizon Patent And Licensing Inc. Evolved packet core (EPC) network failure prevention
CN102811261B (zh) * 2012-08-27 2014-04-02 腾讯科技(深圳)有限公司 信息传输方法、装置、***、终端及服务器
CN103098025A (zh) * 2012-10-24 2013-05-08 华为技术有限公司 软件加载处理方法、装置和网络***
US9565620B2 (en) * 2014-09-02 2017-02-07 Mueller International, Llc Dynamic routing in a mesh network
US9882959B2 (en) * 2014-09-30 2018-01-30 International Business Machines Corporation Managing requests to a high utilization website

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140140189A1 (en) * 2002-05-14 2014-05-22 Genghiscomm Holdings, LLC Cooperative Subspace Demultiplexing in Communication Networks
CN102045400A (zh) * 2010-12-29 2011-05-04 北京世纪互联工程技术服务有限公司 一种文件内容同步方法和***
CN103634346A (zh) * 2012-08-23 2014-03-12 腾讯科技(深圳)有限公司 网络数据中心的数据分发方法和***
CN103546546A (zh) * 2013-09-30 2014-01-29 蓝盾信息安全技术股份有限公司 一种大规模集群分发文件的方法
CN104317716A (zh) * 2014-10-30 2015-01-28 华为技术有限公司 分布式节点间的数据传输方法及分布式节点设备

Also Published As

Publication number Publication date
CN106453460A (zh) 2017-02-22
CN106453460B (zh) 2021-01-08
US20180013610A1 (en) 2018-01-11

Similar Documents

Publication Publication Date Title
WO2017024805A1 (zh) 一种文件分发方法、装置和***
WO2014010992A1 (ko) 컨텐츠 이름 기반의 컨텐츠 중심 네트워크에서 컨텐츠 및 실시간 스트리밍 컨텐츠 제공을 위한 컨텐츠 요청자 및 컨텐츠 제공자의 통신 방법
WO2014081264A1 (en) Method of packet transmission from node and contentowner in content-centric networking
WO2016023148A1 (zh) 报文的控制方法、交换机及控制器
WO2018038490A1 (ko) 무선 통신 네트워크에서 지역별 데이터 네트워크 구성을 위한 방법 및 시스템
EP3017395A1 (en) Method and apparatus of data authentication
WO2018076812A1 (zh) 数据请求的响应方法、装置、存储介质、服务器及***
WO2018076873A1 (zh) 数据分享方法、装置、介质、电子设备及***
WO2017092268A1 (zh) 终端故障处理方法、装置及***
WO2014015809A1 (zh) Ugc主备数据同步方法及其***、计算机存储介质
WO2018076840A1 (zh) 数据分享方法、装置、存储介质及服务器
WO2016065705A1 (zh) 一种频道列表更新方法、装置及终端设备
WO2016065619A1 (zh) 一种数据流量管理方法及装置
WO2018040421A1 (zh) 资源共享方法、装置及***
WO2014019139A1 (zh) 一种实现无线资源控制连接释放的方法、装置及***
WO2012124985A2 (en) Method and apparatus for synchronizing personal inforamtion
WO2013178003A1 (zh) 服务节点切换方法及***
WO2020071809A1 (en) Method and apparatus for enhanced assertion management in cloud media processing
WO2013107212A1 (zh) 一种文件下载方法、装置及***
WO2018076829A1 (zh) 终端数据的处理方法、装置、***、存储介质及服务器
WO2017084301A1 (zh) 音频数据播放方法、装置及智能电视机
WO2013183944A1 (en) Apparatus and method for transmitting and receiving files in general purpose device
WO2017185279A1 (zh) 一种嵌入分布式组网方法及其***
WO2012124999A2 (ko) 단말의 리소스 제공 방법 및 서버의 리소스 획득 방법
EP3443806A1 (en) Method for performing light connection control on user equipment and corresponding equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16834431

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16834431

Country of ref document: EP

Kind code of ref document: A1