CN111866176A - FTP/FTPS distributed file transmission method and device and computer storage medium - Google Patents
FTP/FTPS distributed file transmission method and device and computer storage medium Download PDFInfo
- Publication number
- CN111866176A CN111866176A CN202010773644.XA CN202010773644A CN111866176A CN 111866176 A CN111866176 A CN 111866176A CN 202010773644 A CN202010773644 A CN 202010773644A CN 111866176 A CN111866176 A CN 111866176A
- Authority
- CN
- China
- Prior art keywords
- file
- client
- ftp
- data nodes
- ftps
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000005540 biological transmission Effects 0.000 title claims abstract description 24
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 238000012546 transfer Methods 0.000 claims description 19
- 238000002377 Fourier profilometry Methods 0.000 claims 4
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000009826 distribution Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a device for transmitting FTP/FTPS distributed files and a computer storage medium, which relate to the technical field of internet, and the method comprises the following steps: the client is linked to the control end by using an FTP/FTPS protocol; the client side generates a file transmission request according to a file required to be transmitted by the operation of a user; after receiving the file transmission request, the control end determines a plurality of data nodes which are close to the client, forms a list of the information of the data nodes and feeds the list back to the client; the client establishes a corresponding relation between the data nodes and the file blocks in the list, and transmits the file blocks to each data node according to the corresponding relation, or transmits the file blocks to the client according to the corresponding relation by the data nodes. The method of the invention can use a reliable, rapid and safe mode to transmit the file, and simultaneously uses the distributed FTP service to meet the high-requirement file transmission service required in the software deployment process.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a method and a device for transmitting FTP/FTPS distributed files and a computer storage medium.
Background
In increasingly complex network environments, deploying and publishing application services becomes increasingly difficult, with storage and distribution of files becoming increasingly complex. For example: in multi-regional software deployment, the problems of software version consistency, interface consistency and the like in each region are designed, and the following steps are included: when multiple services are deployed simultaneously, the problem of dependency among the services is solved. Various problems encountered in actual production place higher demands on software deployment.
From the aspect of file distribution, the FTP protocol can perform reliable file transfer, but the protocol supports distributed file transfer poorly. For example: block verification and encryption of files, distributed file authority verification, distributed servers and the like. At present, the FTP protocol cannot well meet the requirements of software rapid release and fault rapid rollback. Other protocols such as the BT protocol, which is prohibited in many network environments and which does not fully support file security, and the SSH protocol. The SSH protocol does not meet the requirements for system security and network penetration.
Disclosure of Invention
The embodiment of the invention provides a method and a device for transmitting FTP/FTPS distributed files and a computer storage medium, which can solve the problems in the prior art.
The invention provides a FTP/FTPS distributed file transmission method, which comprises the following steps:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects files to be transmitted according to the operation of a user and then generates a file transmission request;
after receiving the file transmission request, the control end determines a plurality of data nodes which are close to the client, forms a list of the information of the data nodes and feeds the list back to the client;
the client establishes a corresponding relation between the data nodes and the file blocks in the list, and transmits the file blocks to each data node according to the corresponding relation, or transmits the file blocks to the client according to the corresponding relation by the data nodes.
Preferably, when the file transfer is a file upload, the method comprises the steps of:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects files to be uploaded according to the operation of a user and then generates a file uploading request;
after receiving a file uploading request, the control end determines a plurality of data nodes which are close to the client, forms a list of information of the data nodes and feeds the list back to the client;
the client establishes a corresponding relation between the data nodes and the file blocks in the list, divides the file to be uploaded into a plurality of file blocks, and uploads the file blocks to each data node according to the corresponding relation.
Preferably, after the data node saves the file block, the saved file block is encrypted, and the security and tamper resistance of the file block are improved in a way that the data node saves the HASH value of the file block.
Preferably, when the file transfer is a file download, the method comprises the steps of:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects a file to be downloaded according to the operation of a user, and then generates a request for downloading the file to be viewed;
after receiving the file downloading request, the control end determines a plurality of data nodes which are close to the client, forms a list of the information of the data nodes and feeds the list back to the client;
the client determines the file blocks needing to be downloaded according to the corresponding relation between the files and the file blocks, then judges whether the file blocks needing to be downloaded are stored in the data nodes in the list or not, if yes, the file blocks are downloaded to the client according to the corresponding relation between the file blocks and the data nodes, and the client combines the downloaded file blocks to form a complete file.
Preferably, the method for the control end to determine the plurality of data nodes close to the client includes:
after receiving the file transmission request, determining the position of the client in the network, and then determining a plurality of data nodes close to the client according to the pre-stored position information of each data node in each network.
The invention also provides a device for FTP/FTPS distributed file transmission, which comprises a client, a control end and a data node, wherein the client, the control end and the data node are used for implementing the steps of the file transmission method.
The invention also provides a computer storage medium, wherein the computer storage medium stores a file transmission program, and the file transmission program realizes the steps of the file transmission method when being executed by a processor.
The FTP/FTPS distributed file transmission method, the FTP/FTPS distributed file transmission device and the computer storage medium have the advantages that:
the method can use a reliable, quick and safe mode to transmit the files, and simultaneously uses the distributed FTP service to meet the high-requirement file transmission service required in the software deployment process, thereby improving the quick release and rollback performance of the files.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic diagram of a network environment for implementing a file transfer method according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a network environment implemented by the file transfer method of the present invention includes a network a and a network B, each of which is configured with a plurality of service nodes, where the service nodes include two types: control end and data node. Each service node may be deployed in different networks, where at least one data node needs to be deployed in each network to store a file block. However, there are some obstacles to the file transmission between different networks, such as limitations in ACL and QoS, and the method of the present invention is intended to solve the problems of file transmission between different networks.
Specifically, when the service node is deployed, multiple deployment manners may be adopted, for example, one control end and three data nodes may be deployed in one network, or three control ends and five data nodes may also be deployed in one network. The control terminal for performing the inter-network file transfer service may be in the a network, the B network in fig. 1, or in other networks.
The FTP/FTPS distributed file transmission method comprises two processes of file uploading and file downloading, wherein the file uploading process comprises the following steps:
step 1, the client is linked to the control end by using FTP/FTPS protocol.
And 2, the client selects a corresponding directory and files in the directory according to the operation of the user, and then generates a file uploading request in the form of a PASV command.
And 3, after receiving the file uploading request, the control terminal determines the position of the client in the network, then determines a plurality of data nodes close to the client according to the pre-stored position information of each data node in each network, forms a list of the information of the data nodes and feeds the list back to the client.
And 4, the client establishes a corresponding relation between the data nodes and the file blocks in the list according to the operation of the user, divides the file to be uploaded into a plurality of file blocks, and uploads the file blocks to each data node according to the corresponding relation. After the data node stores the file block, the stored file block is encrypted, and the security and the anti-tampering capability of the file block are improved in a mode that the data node stores the HASH value of the file block.
And 5, after the file is uploaded, managing the link by the control end, such as link establishment, link removal, connection maintenance and the like.
The file downloading process comprises the following steps:
step 1, the client is linked to the control end by using FTP/FTPS protocol.
And 2, the client selects the corresponding directory and the files in the directory according to the operation of the user, and then generates a file downloading request.
And 3, after receiving the file downloading request, the control terminal determines the position of the client in the network, then determines a plurality of data nodes close to the client according to the pre-stored position information of each data node in each network, forms a list of the information of the data nodes and feeds the list back to the client.
And 4, the client determines the file blocks needing to be downloaded according to the corresponding relation between the file and the file blocks, then judges whether the file blocks needing to be downloaded are stored in the data nodes in the list, if so, the file blocks are downloaded to the client according to the corresponding relation between the file blocks and the data nodes, and the client combines the downloaded file blocks to form a complete file.
And 5, after the file is downloaded, managing the link by the control end, such as link establishment, link removal, connection maintenance and the like.
Based on the same inventive concept, the invention provides an FTP/FTPS distributed file transmission device, which comprises a client, a control end and a plurality of data nodes.
Based on the same inventive concept, the invention also provides a computer storage medium for FTP/FTPS distributed file transfer, wherein a file transfer program is stored in the computer storage medium, and the file transfer program realizes the steps of the file transfer method when being executed by a processor.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (7)
1. A FTP/FTPS distributed file transfer method is characterized by comprising the following steps:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects files to be transmitted according to the operation of a user and then generates a file transmission request;
after receiving the file transmission request, the control end determines a plurality of data nodes which are close to the client, forms a list of the information of the data nodes and feeds the list back to the client;
the client establishes a corresponding relation between the data nodes and the file blocks in the list, and transmits the file blocks to each data node according to the corresponding relation, or transmits the file blocks to the client according to the corresponding relation by the data nodes.
2. The FTP/FTPs distributed file transfer method of claim 1, wherein when the file transfer is a file upload, the method comprises the steps of:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects files to be uploaded according to the operation of a user and then generates a file uploading request;
after receiving a file uploading request, the control end determines a plurality of data nodes which are close to the client, forms a list of information of the data nodes and feeds the list back to the client;
the client establishes a corresponding relation between the data nodes and the file blocks in the list, divides the file to be uploaded into a plurality of file blocks, and uploads the file blocks to each data node according to the corresponding relation.
3. The FTP/FTPS distributed file transfer method of claim 2, wherein after the data node saves the file block, the saved file block is encrypted, and the security and tamper resistance of the file block are improved by the way that the data node saves the HASH value of the file block.
4. The FTP/FTPs distributed file transfer method of claim 1, wherein when the file transfer is a file download, the method comprises the steps of:
the client is linked to the control end by using an FTP/FTPS protocol;
the client selects a file to be downloaded according to the operation of a user, and then generates a request for downloading the file to be viewed;
after receiving the file downloading request, the control end determines a plurality of data nodes which are close to the client, forms a list of the information of the data nodes and feeds the list back to the client;
the client determines the file blocks needing to be downloaded according to the corresponding relation between the files and the file blocks, then judges whether the file blocks needing to be downloaded are stored in the data nodes in the list or not, if yes, the file blocks are downloaded to the client according to the corresponding relation between the file blocks and the data nodes, and the client combines the downloaded file blocks to form a complete file.
5. The FTP/FTPs distributed file transfer method of claim 1, wherein the method for the control end to determine the plurality of data nodes that are close to the client is as follows:
after receiving the file transmission request, determining the position of the client in the network, and then determining a plurality of data nodes close to the client according to the pre-stored position information of each data node in each network.
6. An FTP/FTPs distributed file transfer apparatus, comprising a client, a control end and a data node, wherein the client, the control end and the data node are configured to implement the steps of the file transfer method according to any one of claims 1 to 5.
7. A computer storage medium, characterized in that a file transfer program is stored in the computer storage medium, which when executed by a processor implements the steps of the file transfer method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773644.XA CN111866176A (en) | 2020-08-04 | 2020-08-04 | FTP/FTPS distributed file transmission method and device and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010773644.XA CN111866176A (en) | 2020-08-04 | 2020-08-04 | FTP/FTPS distributed file transmission method and device and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111866176A true CN111866176A (en) | 2020-10-30 |
Family
ID=72953498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010773644.XA Pending CN111866176A (en) | 2020-08-04 | 2020-08-04 | FTP/FTPS distributed file transmission method and device and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111866176A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114338700A (en) * | 2021-12-23 | 2022-04-12 | 杭州星布智能科技有限公司 | Local area network point-to-point distributed storage method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013097674A1 (en) * | 2011-12-28 | 2013-07-04 | 北京奇虎科技有限公司 | Distributed system and data operation method thereof |
| CN105162878A (en) * | 2015-09-24 | 2015-12-16 | 网宿科技股份有限公司 | Distributed storage based file distribution system and method |
| CN105635287A (en) * | 2015-12-31 | 2016-06-01 | 深圳市网心科技有限公司 | File transmission system and method |
| CN106487768A (en) * | 2015-08-31 | 2017-03-08 | 北京国双科技有限公司 | A kind of file sharing method and device |
| CN110198330A (en) * | 2018-03-27 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Image download method and apparatus |
-
2020
- 2020-08-04 CN CN202010773644.XA patent/CN111866176A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013097674A1 (en) * | 2011-12-28 | 2013-07-04 | 北京奇虎科技有限公司 | Distributed system and data operation method thereof |
| CN106487768A (en) * | 2015-08-31 | 2017-03-08 | 北京国双科技有限公司 | A kind of file sharing method and device |
| CN105162878A (en) * | 2015-09-24 | 2015-12-16 | 网宿科技股份有限公司 | Distributed storage based file distribution system and method |
| CN105635287A (en) * | 2015-12-31 | 2016-06-01 | 深圳市网心科技有限公司 | File transmission system and method |
| CN110198330A (en) * | 2018-03-27 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Image download method and apparatus |
Non-Patent Citations (6)
| Title |
|---|
| ABELROSE: "【Hadoop】— HDFS(Hadoop分布式文件***)", 《CSDN博客》 * |
| ABELROSE: "【Hadoop】— HDFS(Hadoop分布式文件***)", 《CSDN博客》, 24 July 2020 (2020-07-24), pages 1 - 13 * |
| JSTARSEVEN: ""Hadoop(六)之HDFS的存储原理(运行原理)", 《博客园》 * |
| JSTARSEVEN: ""Hadoop(六)之HDFS的存储原理(运行原理)", 《博客园》, 17 October 2017 (2017-10-17), pages 1 - 9 * |
| 天健胡马灵越鸟: "关于分布式存储，这是你应该知道的（图文详解）（关于存储的一些好文转载--1）", 《CSDN博客》, 1 November 2018 (2018-11-01), pages 1 - 10 * |
| 韦鹏程等: "《大数据时代hadoop技术及应用分析》", vol. 1, 东北师范大学出版社, pages: 235 - 236 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114338700A (en) * | 2021-12-23 | 2022-04-12 | 杭州星布智能科技有限公司 | Local area network point-to-point distributed storage method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2005239708B2 (en) | Method and system for downloading updates | |
| JP4860070B2 (en) | File transmission method and file transmission system | |
| CN109474606A (en) | Document transmission method, device, computer equipment and storage medium | |
| US10693720B2 (en) | Distribution of updates in an IoT network | |
| CN101399729B (en) | Management system and method for file download and upload | |
| KR20080047353A (en) | Distributed caching of files in a network | |
| JP2009500759A (en) | Method and system for data management based on simple network management protocol | |
| CN107005453B (en) | Method and system for secure two-way communication of industrial devices | |
| CN111654399B (en) | Networking method, device, equipment and storage medium based on SD-WAN | |
| CN109391673A (en) | A kind of method, system and the terminal device of management update file | |
| US8959242B1 (en) | Method of reducing bandwidth required for sharing of files on a server | |
| US20180316747A1 (en) | Managing content downloads | |
| CN111857772B (en) | Method and device for remote transmission and dynamic loading of terminal edge application program | |
| EP3206347B1 (en) | Method for calling a routing algorithm, sdn controller, and sdn-oaf | |
| CN111866176A (en) | FTP/FTPS distributed file transmission method and device and computer storage medium | |
| CN101159676B (en) | Data transmission method and system | |
| CA2595438A1 (en) | Method for improving peer to peer network communication | |
| CN112256797A (en) | MongoDB-based data synchronization system and method for public security network and government affair network | |
| CN115150162B (en) | Root certificate updating method and device | |
| CN101500020B (en) | Method and apparatus for transmitting data block based on point-to-point protocol | |
| CN114338646B (en) | File interactive transmission method, device, equipment and storage medium | |
| JP2006171918A5 (en) | ||
| CN112583926B (en) | Short-link switching method, system, computer equipment and storage medium | |
| CN113542282A (en) | Cross-computing center file transmission method, device, equipment and storage medium | |
| KR101157483B1 (en) | Method for downloading file and record media recorded program for realizing the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201030 |
|
| RJ01 | Rejection of invention patent application after publication |