CN113572813A - Data backup method based on network coding - Google Patents
Data backup method based on network coding Download PDFInfo
- Publication number
- CN113572813A CN113572813A CN202110692824.XA CN202110692824A CN113572813A CN 113572813 A CN113572813 A CN 113572813A CN 202110692824 A CN202110692824 A CN 202110692824A CN 113572813 A CN113572813 A CN 113572813A
- Authority
- CN
- China
- Prior art keywords
- cloud
- data
- coding
- backup
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/104—Peer-to-peer [P2P] networks
- H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
- H04L67/1078—Resource delivery mechanisms
- H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data backup method based on network coding, which comprises the following steps: step 1, determining available network bandwidth and coding parameters between cloud storage points; step 2, making decisions on user cloud network traffic and backup cloud network traffic according to available network bandwidth and coding parameters to obtain an initial cloud candidate set and a coding group set used in a network coding process; and 3, backing up the user cloud data to the interpyury storage system by adopting a network coding mechanism according to the coding group set. The invention designs an efficient network coding scheme, is compatible with various data coding strategies, and adjusts according to the network bandwidth between the cloud storage nodes, so that the network bandwidth resources between the cloud storage nodes are fully utilized, and the method can be applied to an inter-cloud storage system, so as to reduce the influence of the bandwidth bottleneck of a network link on the data backup operation performance, and improve the efficiency of executing large-data-volume backup operation by an online cloud service system.
Description
Technical Field
The invention belongs to the field of data storage, and particularly relates to a data backup method based on network coding.
Background
According to the statistical report of the international well-known data company IDC, the user data generated by the online application shows explosive growth. Data generated by humans doubles every two years since 2020, enterprise data doubles every 6 to 9 months, and 175ZB data is expected to be generated by 2025. As a scheme capable of providing an inexpensive data storage service, the cloud storage system is widely applied to current online applications for storing user data. In the conventional cloud storage system, since all user data is stored in the same cloud service provider, data loss and leakage problems are likely to occur when a cloud storage node fails or is broken by a hacker.
The inter-cloud storage system is used as an efficient online storage solution to ensure the reliability and safety of stored data. By encrypting, encoding and storing user data to a plurality of cloud storage nodes, the inter-cloud storage system can restore original user data when the inter-cloud storage nodes fail. When part of cloud storage nodes are broken by hackers, the interpyury storage system can ensure that the user data cannot be leaked by encrypting the user data.
In order to ensure the reliability of cloud data, a cloud service system generally needs to periodically backup a large amount of user data to another online storage system. However, the inter-cloud storage system has difficulty in ensuring the performance of the data backup operation of the online business system. When a large amount of data needs to be transmitted in the inter-cloud storage system backup process, due to the fact that data needs to be transmitted between different cloud storage nodes through the internet, due to the fact that network bandwidth resources are limited, much time is generally consumed, and large data backup delay is caused. How to design an efficient data transmission mechanism to reduce data transmission overhead in a data backup process becomes a problem to be solved urgently.
Aiming at the problem that the data backup operation of the interpyury storage system consumes longer time, a method capable of solving the problem needs to be designed.
Disclosure of Invention
The present invention is made to solve the above problems, and an object of the present invention is to provide a data backup method based on network coding.
The invention provides a data backup method based on network coding, which is characterized by comprising the following steps: step 1, determining available network bandwidth and coding parameters between cloud storage points; step 2, making decisions on user cloud network traffic and backup cloud network traffic according to available network bandwidth and coding parameters to obtain an initial cloud candidate set and a coding group set used in a network coding process; and 3, backing up the user cloud data to the interpyury storage system by adopting a network coding mechanism according to the coding group set.
The data backup method based on network coding provided by the invention can also have the following characteristics: the user cloud network flow decision in the step 2 comprises the following substeps: step S2-1, calculating the minimum transmission data volume which needs to be uploaded to the backup cloud by the user cloud according to the encoding parameters; step S2-2, calculating the minimum uploading time required by the user cloud to complete the data backup process under the condition of the minimum transmission data volume according to the available network bandwidth from the user cloud to the backup cloud; step S2-3, on the premise that the minimum uploading time of the user cloud is not changed, increasing the data volume uploaded to the backup cloud by the user cloud; step S2-4, for each strip, generating an initial cloud candidate set according to the data volume of the backup cloud to be uploaded, wherein the backup cloud network traffic decision in step 2 comprises the following substeps: step T2-1, for each strip, distributing an initial cloud to each subsequent cloud according to the available network bandwidth between the backup clouds and the initial cloud candidate set, and ensuring that a single subsequent cloud generates check data within the minimum time; step T2-2, when a plurality of subsequent clouds have the same initial cloud, checking whether the time consumed by data backup operation is reduced by exchanging data among the subsequent clouds, and when the data consumed by data backup operation is reduced, merging the subsequent clouds into a coding group to allow the subsequent clouds in the coding group to exchange data for reducing the data backup time, otherwise, keeping the original transmission scheme unchanged; and T2-3, scanning all the strips, and generating a coding group set used in the network coding process according to the merging result of the initial cloud and the subsequent cloud.
The data backup method based on network coding provided by the invention can also have the following characteristics: and 3, the step comprises a user cloud data coding transmission process and a backup cloud network coding transmission process.
The data backup method based on network coding provided by the invention can also have the following characteristics: the user cloud data coding and transmitting process specifically comprises the following substeps: s3-1, encrypting the user message to be stored by adopting a data encryption algorithm to obtain encrypted information; step S3-2, generating a network flow and a coding group set which need to be transmitted between cloud storage nodes according to the available network bandwidth between the user cloud and the backup cloud and the coding parameters of the inter-cloud storage system; step S3-3, according to the coding rule and the coding group set, coding the encrypted message to generate the fragment data and the packet data; and step S3-4, uploading the fragment data and the packet data to a backup cloud by the user cloud according to the network traffic and the coding group set.
The data backup method based on network coding provided by the invention can also have the following characteristics: wherein, the data encryption algorithm in the step S3-1 adopts an ANOT encryption algorithm.
The data backup method based on network coding provided by the invention can also have the following characteristics: the backup cloud network coding transmission process specifically comprises the following substeps: step T3-1, for a given stripe, the backup cloud defines the network coding operation to be executed in the network coding process according to the coding group set received from the user cloud; step T3-2, judging the cloud type of the backup cloud, when the backup cloud is an initial cloud, receiving the packet data from the user side by the initial cloud according to the information of the coding group set, then forwarding the received packet data to a subsequent cloud in the group according to the network coding rule, when the backup cloud is the subsequent cloud, downloading the packet data from the initial cloud in the same group by the subsequent cloud, then calculating to generate check data according to the network coding rule, exchanging the check data with other subsequent cloud nodes in the group, then combining the coded data and the received data to generate the check data in the strip, thereby completing the network coding operation of all backup clouds in the strip; and T3-3, repeating the steps T3-1 and T3-2, completing network coding operation of all backup clouds in all the stripes, obtaining check data in all the stripes, and merging the check data to obtain fragment data.
Action and Effect of the invention
Aiming at the problem that a large amount of time is consumed in the data backup process in the interpyury storage system, the data backup method based on the network coding introduces a network coding scheme to accelerate the data backup process; in order to quickly transmit data of a user cloud to a plurality of backup clouds, the invention can provide a data backup method based on network coding to adjust the data volume transmitted between cloud storage nodes according to the available network bandwidth of an interpyury storage system; in order to ensure that the proposed algorithm can be compatible with various interpyury storage systems, the invention constructs an efficient data coding method, and executes network coding operation under the condition of not changing the coding rule of the interpyury storage system; in addition, the transmission data of the network coding is divided into a plurality of strips and coding groups, and data exchange among the cloud storage nodes is allowed, so that the method can fully utilize network resources among the cloud storage nodes, then on the basis, the cloud storage nodes are divided into an initial cloud and a subsequent cloud, and the node routing of the transmission data is effectively controlled through the method.
Therefore, the invention designs an efficient network coding scheme, is compatible with various data coding strategies, and adjusts according to the network bandwidth between the cloud storage nodes, thereby fully utilizing the network bandwidth resources between the cloud storage nodes, being applicable to an inter-cloud storage system, reducing the influence of the network link bandwidth bottleneck on the data backup operation performance, and improving the efficiency of executing the large-data-volume backup operation by the online cloud service system.
Drawings
Fig. 1 is a flowchart of a user cloud network traffic decision process of the network coding-based data backup method of the present invention;
FIG. 2 is a flow chart of a decision process of network traffic between backup clouds of the network coding-based data backup method of the present invention;
FIG. 3 is a flow chart of a data backup process of the network coding based data backup method of the present invention;
FIG. 4 is a schematic diagram of a network topology between cloud storage nodes in a distributed storage system bottleneck link in an embodiment of the invention;
FIG. 5 is a diagram illustrating a data backup method for a conventional inter-cloud storage system in a bottleneck link of a distributed storage system according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a method for backing up data based on network coding in a bottleneck link of a distributed storage system according to an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating a relationship between stripes and coding groups in an example of a decision of stripes and coding groups in a data backup process according to an embodiment of the present invention;
fig. 8 is a schematic diagram of an execution process of network coding in a coding group in an example of decision of a stripe and a coding group in a data backup process in an embodiment of the present invention.
Detailed Description
In order to make the technical means and functions of the present invention easy to understand, the present invention is specifically described below with reference to the embodiments and the accompanying drawings.
The invention provides a data backup method based on network coding, which comprises the following steps:
And 2, making decisions on the user cloud network flow and the backup cloud network flow according to the available network bandwidth and the coding parameters to obtain an initial cloud candidate set and a coding group set used in the network coding process.
As shown in fig. 1, the user cloud network traffic decision in the present invention includes the following sub-steps:
step S2-1, calculating the minimum transmission data volume which needs to be uploaded to the backup cloud by the user cloud according to the encoding parameters;
step S2-2, calculating the minimum uploading time required by the user cloud to complete the data backup process under the condition of the minimum transmission data volume according to the available network bandwidth from the user cloud to the backup cloud;
step S2-3, on the premise that the minimum uploading time of the user cloud is not changed, increasing the data volume uploaded to the backup cloud by the user cloud;
and step S2-4, generating an initial cloud candidate set according to the data volume of the backup cloud to be uploaded for each coding group.
The cloud in the stripe receiving data from the user side is called an initial cloud, and the cloud receiving data from other cloud storage nodes is called a subsequent cloud.
As shown in fig. 2, the backup cloud network traffic decision in the present invention includes the following sub-steps:
step T2-1, for each coding group, distributing an initial cloud to each subsequent cloud according to the available network bandwidth between the backup clouds and the initial cloud candidate set, and ensuring that a single subsequent cloud generates verification data within the minimum time;
step T2-2, when a plurality of subsequent clouds have the same initial cloud, checking whether the time consumed by data backup operation is reduced by exchanging data among the subsequent clouds, and when the data consumed by data backup operation is reduced, merging the subsequent clouds into a coding group to allow the subsequent clouds in the coding group to exchange data for reducing the data backup time;
and T2-3, scanning all the strips, and generating a coding group set used in the network coding process according to the merging result of the initial cloud and the subsequent cloud.
And 3, adopting a network coding mechanism and backing up and copying the user cloud data to the inter-cloud storage system according to the coding group set.
As shown in fig. 3, step 3 in the present invention includes a user cloud data encoding and transmitting process and a backup cloud network encoding and transmitting process.
Specifically, the user cloud data encoding and transmitting process specifically includes the following sub-steps:
step S3-1, encrypting the user message to be stored by adopting a data encryption algorithm to obtain encryption information, wherein the data encryption algorithm adopts an ANOT encryption algorithm, namely an All Or Nothing encryption algorithm;
step S3-2, generating a network flow and a coding group set which need to be transmitted between cloud storage nodes according to the available network bandwidth between the user cloud and the backup cloud and the coding parameters of the inter-cloud storage system;
step S3-3, according to the coding rule and the coding group set, coding the encrypted message to generate the fragment data and the packet data;
and step S3-4, uploading the fragment data and the packet data to a backup cloud by the user cloud according to the network traffic and the coding group set.
Meanwhile, the backup cloud network coding transmission process specifically comprises the following substeps:
step T3-1, for a given stripe, the backup cloud defines the network coding operation to be executed in the network coding process according to the coding group set received from the user cloud;
step T3-2, judging the cloud type of the backup cloud, when the backup cloud is an initial cloud, receiving the packet data from the user side by the initial cloud according to the information of the coding group set, then forwarding the received packet data to a subsequent cloud in the group according to the network coding rule, when the backup cloud is the subsequent cloud, downloading the packet data from the initial cloud in the same group by the subsequent cloud, then calculating to generate check data according to the network coding rule, exchanging the check data with other subsequent cloud nodes in the group, then combining the coded data and the received data to generate the check data in the strip, thereby completing the network coding operation of all backup clouds in the strip;
and T3-3, repeating the steps T3-1 and T3-2, completing network coding operation of all backup clouds in all the stripes, obtaining check data in all the stripes, and merging the check data to obtain fragment data.
The invention can effectively improve the efficiency of backing up data to the interpyury storage system by an online cloud service provider, and can be deployed to the following online service application scenes:
(1) and the high-reliability inter-cloud storage backup scene. In an online service scenario where high reliability is required, data backup operations need to be completed within a given time. By deploying the data backup method based on network coding provided by the invention, an online application service provider can effectively control the time consumed by data backup operation, thereby ensuring the reliability and efficiency of stored data.
(2) And the large data volume interpolant stores backup scenes. Current online applications generate large amounts of user data, which often takes a long time to backup to the inter-cloud storage system. The data backup method based on the network coding can fully utilize the network bandwidth among the cloud storage nodes, reduce the time consumed by executing data backup operation and improve the availability of online application service.
Example (b):
the user cloud needs to backup data to the interplant storage system, and the data backup comprises k 2 backup cloud storage data fragments and m 2 backup cloud storage verification fragments. Fig. 4-6 illustrate the implementation of various data backup methods in an interpyury storage system.
Fig. 4 illustrates an inter-cloud storage system comprising a user cloud and four backup clouds. The user cloud needs to backup the M-byte file to the backup cloud. Fig. 5 shows a data backup method adopted by the current inter-cloud storage system. The user cloud divides the data into a data fragment A and a data fragment B, and codes to generate a check fragment A + B and a check fragment A + 2B. Each slice is of a size of bytes of data. In this example, the inter-cloud storage system takes seconds to complete the data backup operation.
By analyzing the data backup process, it can be found that the data backup operation is affected by bottleneck bandwidth between nodes. In order to ensure the execution efficiency of the data backup operation, the present embodiment proposes a data backup method based on network coding to fully utilize network bandwidth resources between cloud storage nodes. Fig. 6 shows the implementation of the method.
First, the user cloud divides data slice a into package a1 and package B2 check slice a +2 × B into package a1+2 × B1 and package a2+2 × B2. Where each packet size is in bytes. When the cloud storage node receives the data fragment B, the data fragment B is divided into a packet B1 and a packet B2, and the check fragment A + B is divided into a packet A1+ B1 and a2+ B2. By exchanging packets B1 and a1+ B1 between cloud storage nodes, the inter-cloud storage system may generate shards a and a +2 × B. In this example, the inter-cloud storage system only requires seconds to complete the data backup operation. Compared with the conventional data backup method, the embodiment can reduce the backup time by half.
As can be seen from the above examples, the data backup method based on network coding can effectively reduce the data backup time by adjusting the amount of data transmitted between nodes. In order to efficiently decide network traffic among cloud storage nodes and construct network codes, according to the coding relationship of the fragments in the cloud storage nodes, the embodiment divides packets to be coded into a plurality of strips and coding group sets, and divides the cloud storage nodes into an initial cloud and a subsequent cloud.
Fig. 7 and 8 show the process of deciding network traffic based on the network coding-based data backup method in the examples of fig. 4-6. Fig. 7 illustrates the slices and coding groups referred to in the examples of fig. 4-6. A group of packets for which there is an encoding relationship is referred to herein as a slice. In a given stripe, the cloud storage nodes generate required check data according to the packet codes in the network coding rule exchange group.
Meanwhile, the data backup method based on the network coding defines an initial cloud and a subsequent cloud to describe the data transmission operation of the network coding process. Fig. 8 shows a process in which the encoding group G1 and the encoding group G2 perform a network encoding operation. In the embodiment, the cloud receiving data from the user side in the coding group is called an initial cloud, and the cloud receiving data from other cloud storage nodes is called a subsequent cloud. For stripe 1, backup cloud C2 and backup cloud C3 are the initial clouds and backup cloud C1 and backup cloud C4 are the successor clouds. For each stripe, the subsequent cloud needs to receive at least k 2 data of the initial cloud to generate the check data.
Effects and effects of the embodiments
According to the embodiment, the data volume uploaded to the backup cloud by the user cloud and the data volume transmitted between the backup clouds need to be determined according to the network bandwidth between the cloud storage nodes and the data backup method based on the network coding.
Further, since the user cloud needs to generate the decision code generation packets and send the decision code generation packets to each cloud storage node, the inter-cloud storage system needs to generate the number of packets sent by the user cloud to each backup cloud. The user cloud can calculate the minimum time required by the user cloud to the backup cloud for transmitting the packet data to complete the data backup operation according to the available network bandwidth and the number of the fragment segmentation packets. In order to fully utilize the network bandwidth from the user cloud to the backup cloud, the user cloud uploads as many packets as possible to the backup cloud on the premise of not increasing the uploading time.
On this basis, data needs to be exchanged between backup clouds to generate check fragments. According to the encoding rule, an initial cloud is distributed to each subsequent cloud, and the verification data generation time of each subsequent cloud is minimized. Since the consumption amount of network bandwidth can be reduced by exchanging data between the subsequent clouds, it is necessary to further merge the subsequent clouds to accelerate the execution efficiency of the data backup operation. In the case where the subsequent cloud has the same initial cloud and data backup time can be reduced by exchanging data, the inter-cloud storage system storage will merge the subsequent cloud to generate the encoding group.
Further, according to a network traffic method between the cloud storage nodes, the inter-cloud storage system executes a network coding method to generate the check data. And according to the available bandwidth resources and the coding information, calculating coding group information in the user cloud, coding user data, and uploading the group information and the coded user data to the backup cloud. For a given stripe, the backup cloud may perform a network coding process as an initial cloud or a subsequent cloud, and then perform a corresponding network coding operation according to the grouping information received from the user cloud. When acting as an initial cloud, the backup cloud receives encoded data from the user cloud and forwards these received data to a subsequent cloud. When the backup cloud is used as a subsequent cloud, the backup cloud receives data from other cloud storage nodes and generates check data according to the encoding rule. And after the inter-cloud storage system finishes the coding operation of all the strips, the backup cloud generates fragments according to the coding result and stores the fragments in the local storage device.
In summary, the data backup method based on network coding of the invention introduces a network coding scheme to accelerate the data backup process aiming at the problem that the data backup process in the interpyury storage system needs to consume more time; in order to quickly transmit data of a user cloud to a plurality of backup clouds, the invention can provide a data backup method based on network coding to adjust the data volume transmitted between cloud storage nodes according to the available network bandwidth of an interpyury storage system; in order to ensure that the proposed algorithm can be compatible with various interpyury storage systems, the invention constructs an efficient data coding method, and executes network coding operation under the condition of not changing the coding rule of the interpyury storage system; in addition, the transmission data of the network coding is divided into a plurality of strips and coding groups, and data exchange among the cloud storage nodes is allowed, so that the method can fully utilize network resources among the cloud storage nodes, then on the basis, the cloud storage nodes are divided into an initial cloud and a subsequent cloud, and the node routing of the transmission data is effectively controlled through the method.
Therefore, the invention designs an efficient network coding scheme, is compatible with various data coding strategies, and adjusts according to the network bandwidth between the cloud storage nodes, thereby fully utilizing the network bandwidth resources between the cloud storage nodes, being applicable to an inter-cloud storage system, reducing the influence of the network link bandwidth bottleneck on the data backup operation performance, and improving the efficiency of executing the large-data-volume backup operation by the online cloud service system.
The above embodiments are preferred examples of the present invention, and are not intended to limit the scope of the present invention.
Claims (6)
1. A data backup method based on network coding is characterized by comprising the following steps:
step 1, determining available network bandwidth and coding parameters between cloud storage points;
step 2, making decisions on user cloud network traffic and backup cloud network traffic according to the available network bandwidth and the coding parameters to obtain an initial cloud candidate set and a coding group set used in a network coding process;
and 3, backing up the user cloud data to an interpyury storage system by adopting a network coding mechanism according to the coding group set.
2. The method for backing up data based on network coding of claim 1, wherein:
the user cloud network traffic decision in step 2 includes the following substeps:
step S2-1, calculating the minimum transmission data volume which needs to be uploaded to the backup cloud by the user cloud according to the encoding parameters;
step S2-2, calculating the minimum uploading time required by the user cloud to complete the data backup process under the condition of the minimum transmission data volume according to the available network bandwidth from the user cloud to the backup cloud;
step S2-3, on the premise that the minimum uploading time of the user cloud is not changed, increasing the data volume uploaded to the backup cloud by the user cloud;
step S2-4, for each stripe, according to the data volume of the backup cloud to be uploaded, generating an initial cloud candidate set,
the backup cloud network traffic decision in the step 2 comprises the following substeps:
step T2-1, for each strip, allocating an initial cloud to each subsequent cloud according to the available network bandwidth between the backup clouds and the initial cloud candidate set, and ensuring that a single subsequent cloud generates check data within the minimum time;
step T2-2, when a plurality of the subsequent clouds have the same initial cloud, checking whether the time consumed by data backup operation is reduced by exchanging data among the subsequent clouds, and when the data consumed by data backup operation is reduced, merging the subsequent clouds into an encoding group to allow the subsequent clouds in the encoding group to exchange data for reducing the data backup time, otherwise, keeping the original transmission scheme unchanged;
and T2-3, scanning all the strips, and generating a coding group set used in the network coding process according to the merging result of the initial cloud and the subsequent cloud.
3. The method for backing up data based on network coding of claim 1, wherein:
and 3, the step comprises a user cloud data coding transmission process and a backup cloud network coding transmission process.
4. The network coding-based data backup method according to claim 3, wherein:
the user cloud data coding and transmitting process specifically comprises the following substeps:
s3-1, encrypting the user message to be stored by adopting a data encryption algorithm to obtain encrypted information;
step S3-2, generating a network flow and a coding group set which need to be transmitted between cloud storage nodes according to the available network bandwidth between the user cloud and the backup cloud and the coding parameters of the inter-cloud storage system;
step S3-3, according to the coding rule and the coding group set, coding the encrypted message to generate the fragment data and the packet data;
and step S3-4, according to the network flow and the coding group set, the user cloud uploads the fragment data and the packet data to a backup cloud.
5. The method for backing up data based on network coding of claim 4, wherein:
wherein, the data encryption algorithm in the step S3-1 adopts an ANOT encryption algorithm.
6. The network coding-based data backup method according to claim 3, wherein:
the backup cloud network coding transmission process specifically comprises the following substeps:
step T3-1, for a given stripe, the backup cloud defines the network coding operation to be executed in the network coding process according to the coding group set received from the user cloud;
step T3-2, judging the cloud type of the backup cloud, when the backup cloud is an initial cloud, receiving packet data from a user side by the initial cloud according to information of a coding group set, then forwarding the received packet data to a subsequent cloud in the group according to a network coding rule, when the backup cloud is the subsequent cloud, downloading the packet data from the initial cloud in the same group by the subsequent cloud, then calculating to generate check data according to the network coding rule, exchanging the check data with other subsequent cloud nodes in the group, then merging the coded data and the received data to generate the check data in the band, and thus finishing the network coding operation of all backup clouds in the band;
and T3-3, repeating the steps T3-1 and T3-2, completing network coding operation of all backup clouds in all the stripes, obtaining check data in all the stripes, and merging the check data to obtain fragment data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110692824.XA CN113572813B (en) | 2021-06-22 | 2021-06-22 | Data backup method based on network coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110692824.XA CN113572813B (en) | 2021-06-22 | 2021-06-22 | Data backup method based on network coding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113572813A true CN113572813A (en) | 2021-10-29 |
| CN113572813B CN113572813B (en) | 2022-06-14 |
Family
ID=78162475
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110692824.XA Active CN113572813B (en) | 2021-06-22 | 2021-06-22 | Data backup method based on network coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113572813B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101605148A (en) * | 2009-05-21 | 2009-12-16 | 何吴迪 | The framework method of the parallel system of cloud storage |
| CN102684817A (en) * | 2011-03-10 | 2012-09-19 | 万鑫明 | Digital fountain code based cloud storage system structure and service providing method |
| US20130111262A1 (en) * | 2010-05-03 | 2013-05-02 | Panzura, Inc. | Providing disaster recovery for a distributed filesystem |
| CN103281400A (en) * | 2013-06-18 | 2013-09-04 | 清华大学 | Data segmenting, coding and recovering method used for cloud storage gateway |
| US9940203B1 (en) * | 2015-06-11 | 2018-04-10 | EMC IP Holding Company LLC | Unified interface for cloud-based backup and restoration |
| CN111629028A (en) * | 2020-04-10 | 2020-09-04 | 清华大学 | Data transmission scheduling system for distributed multi-cloud storage |
| CN111679939A (en) * | 2020-06-16 | 2020-09-18 | 江苏超流信息技术有限公司 | Data backup method and system |
-
2021
- 2021-06-22 CN CN202110692824.XA patent/CN113572813B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101605148A (en) * | 2009-05-21 | 2009-12-16 | 何吴迪 | The framework method of the parallel system of cloud storage |
| US20130111262A1 (en) * | 2010-05-03 | 2013-05-02 | Panzura, Inc. | Providing disaster recovery for a distributed filesystem |
| CN102684817A (en) * | 2011-03-10 | 2012-09-19 | 万鑫明 | Digital fountain code based cloud storage system structure and service providing method |
| CN103281400A (en) * | 2013-06-18 | 2013-09-04 | 清华大学 | Data segmenting, coding and recovering method used for cloud storage gateway |
| US9940203B1 (en) * | 2015-06-11 | 2018-04-10 | EMC IP Holding Company LLC | Unified interface for cloud-based backup and restoration |
| CN111629028A (en) * | 2020-04-10 | 2020-09-04 | 清华大学 | Data transmission scheduling system for distributed multi-cloud storage |
| CN111679939A (en) * | 2020-06-16 | 2020-09-18 | 江苏超流信息技术有限公司 | Data backup method and system |
Non-Patent Citations (6)
| Title |
|---|
| JIAJIE SHEN: "《Mobile Cloud-of-Clouds Storage Made Efficient: A Network Coding Based Approach》", 《IEEE》 * |
| JIAJIE SHEN: "《Mobile Cloud-of-Clouds Storage Made Efficient: A Network Coding Based Approach》", 《IEEE》, 5 October 2018 (2018-10-05) * |
| TSAN-CHANG KUO: "Dynamic bandwidth reallocation for NG-SDH networks based on Real-time Traffic Flow Analysis", 《2012 14TH ASIA-PACIFIC NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM (APNOMS)》, 26 November 2012 (2012-11-26) * |
| 毛文波: "云计算的增值服务与"私有云际存储云"", 《华南金融电脑》, no. 02, 10 February 2010 (2010-02-10) * |
| 谢垂益等: "云存储***的分区编码冗余方法研究", 《信息网络安全》 * |
| 谢垂益等: "云存储***的分区编码冗余方法研究", 《信息网络安全》, no. 06, 10 June 2013 (2013-06-10) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113572813B (en) | 2022-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Cabrera et al. | Softwarization and network coding in the mobile edge cloud for the tactile internet | |
| CN105830377A (en) | Dynamic coding for network traffic by fog computing node | |
| Fitzek | On network coded distributed storage: How to repair in a fog of unreliable peers | |
| Li et al. | Towards green IoT networking: Performance optimization of network coding based communication and reliable storage | |
| Altman et al. | Dynamic control of coding in delay tolerant networks | |
| CN110943800A (en) | Method, device and system for sending data packet, storage medium and electronic device | |
| Cacciapuoti et al. | Speeding up future video distribution via channel-aware caching-aided coded multicast | |
| Fiandrotti et al. | Band codes for energy-efficient network coding with application to P2P mobile streaming | |
| Puducheri et al. | Distributed LT codes | |
| CN110190926B (en) | Erasure code updating method and system based on network computing | |
| Fodor et al. | Resilience in live peer-to-peer streaming [Peer-to-Peer Multimedia Streaming] | |
| Bonomi et al. | Practical Byzantine reliable broadcast on partially connected networks | |
| Costa et al. | Robust improvement of the age of information by adaptive packet coding | |
| CN113572813B (en) | Data backup method based on network coding | |
| Xu et al. | I-swifter: improving chunked network coding for peer-to-peer content distribution | |
| Sattiraju et al. | Reliability analysis of a wireless transmission as a repairable system | |
| Mehta et al. | Survey on multimedia transmission using network coding over wireless networks | |
| CN112118316B (en) | Distributed storage data migration method and device | |
| Torre et al. | Network-coded multigeneration protocols in heterogeneous cellular networks | |
| Gerami et al. | Decentralized minimum-cost repair for distributed storage systems | |
| Peralta et al. | Network coding-based next-generation IoT for industry 4.0 | |
| Baguena et al. | RCDP: Raptor-based content delivery protocol for unicast communication in wireless networks for ITS | |
| Jiang et al. | Disruption‐resilient bundle delivery mechanism in space DTNs with partial segments aggregation | |
| US10789380B2 (en) | Intelligent write protection cache (IWPC) in an adaptive private network (APN) | |
| Do-Duy et al. | Finite-length performance comparison of network codes using random vs Pascal matrices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |