CN109246190B - Network addressing method, data editing method, device and storage medium - Google Patents
Network addressing method, data editing method, device and storage medium Download PDFInfo
- Publication number
- CN109246190B CN109246190B CN201810892073.4A CN201810892073A CN109246190B CN 109246190 B CN109246190 B CN 109246190B CN 201810892073 A CN201810892073 A CN 201810892073A CN 109246190 B CN109246190 B CN 109246190B
- Authority
- CN
- China
- Prior art keywords
- data
- hash value
- target data
- hash
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
The application discloses a network addressing method, a data editing method, a device and a computer readable storage medium based on a distributed file system, which are used for receiving a data access request from a requester, determining whether a node can provide target data pointed by the data access request, and when determining that the target data cannot be provided, not returning the target data to the requester. Through the judgment of the target data access authority, the files of the improper contents can be effectively prevented from being diffused, and the adverse social influence caused by the improper contents is reduced.
Description
Technical Field
The present application relates to the field of distributed platform technologies, and in particular, to a network addressing method, a data editing method, an apparatus, and a computer-readable storage medium based on a distributed file system.
Background
With the development of global network economy, internet data, financial industry data, personal information data, and the like have been explosively increased. The traditional file system has insufficient space and limited processing capability, and cannot meet the requirements of high capacity, high reliability, easy expansion and the like. Therefore, in recent years, the distributed file system has been developed rapidly, and due to the characteristics of higher network throughput and decentralization, the network is safe and transparent, and the constraint of decentralization is removed. In the distributed file system, files in the network can never be deleted from the network as long as a person claims the value of the files, and the effect of permanent storage is achieved.
However, permanently preserving this feature is not characterized as an advantage in all scenarios. For example, some people upload inappropriate content through a network, and the content can be widely spread through the network, and the content cannot be prevented from spreading under a distributed system.
Disclosure of Invention
The application provides a network addressing method, a data editing method and a device of a distributed file system and a computer readable storage medium, which can block inappropriate files in the distributed system and reduce the diffusion extent of the inappropriate files.
In order to solve the above technical problem, a first technical solution provided by the present application is to provide a network addressing method based on a distributed file system, including: receiving a data access request from a requester; determining whether the node can provide target data pointed by the data access request; if the target data is determined to be unavailable, the target data is not returned to the requester.
In order to solve the above technical problem, a second technical solution provided by the present application is to provide a data editing method based on a distributed file system, including: receiving an editing request of target data permission from a requester; determining whether the requestor is an owner of the target data; and if the authority is determined to be the owner, the editing request is responded to edit the target data authority, so that whether data can be provided or not is operated according to the edited target data authority when an access request to the target data is subsequently received.
In order to solve the above technical problem, a third technical solution provided by the present application is to provide a data query apparatus based on a distributed file system, including: the data query method comprises the following steps of a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the computer program to realize the steps of the data query method.
In order to solve the above technical problem, a fourth technical solution provided by the present application is to provide a data editing apparatus based on a distributed file system, including: the data editing method comprises the following steps of a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the steps of the data editing method are realized when the processor executes the computer program.
In order to solve the above technical problem, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data query method or the data editing method.
Different from the prior art, the data query method is based on a distributed file system and is used for querying data in the distributed file system; the data query method comprises the following steps: the method comprises the steps of receiving a data access request from a requester, determining whether the target data pointed by the data access request can be provided by the node, and not returning the target data to the requester when the target data is determined not to be provided. Through the judgment of the target data access authority, the files of the improper contents can be effectively prevented from being diffused, and the adverse social influence caused by the improper contents is reduced.
Drawings
FIG. 1 is a schematic diagram of an interaction between a requestor and an embodiment of a network node;
FIG. 2 is a flowchart illustrating an embodiment of a distributed file system based network addressing method according to the present application;
FIG. 3 is a diagram of an embodiment of a Mercker data structure for hash values of target data according to the present application;
FIG. 4 is a diagram illustrating an embodiment of a hash value generation method for target data according to the present application;
FIG. 5 is a schematic interaction diagram of a requestor of the present application with another embodiment of a network node;
FIG. 6 is a schematic flowchart of an embodiment of a distributed file system-based data storage method according to the present application;
FIG. 7 is a flowchart illustrating an embodiment of a distributed file system based data editing method according to the present application;
FIG. 8 is a schematic structural diagram of an embodiment of a data querying device based on a distributed file system according to the present application;
FIG. 9 is a schematic structural diagram of an embodiment of a data editing apparatus based on a distributed file system according to the present application;
FIG. 10 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The distributed file system is an open-source lightweight distributed file system, and comprises various types, such as a FastDFS distributed system, a GFS distributed file system, a TFS distributed file system, an HDFS distributed file system, an interplanetary file system IPFS and the like.
A network node is a computer or other device connected to a network having an independent address and having a function of transmitting or receiving data. Such as workstations, clients, network users, personal computers, servers, printers, or other network-connected devices, etc., the device having its own unique network address may become a network node.
A distributed file system is formed by a plurality of network nodes connected by communication lines. The method and the device mainly limit the query and download of the blocked data information by carrying out blocking setting on the data information in part of network nodes in the distributed file system.
As shown in fig. 1, fig. 1 is an interaction diagram of a requester and a network node for implementing data query. In the communication process between the network node and the requester, the method mainly comprises the following processes that the requester sends a data access request to the network node, the local network node receives the data access request, determines whether the local node can provide target data pointed by the data access request, and if the target data is determined not to be provided, the target data is not returned to the requester. And if the target data can be provided and the node is determined to have the target data, returning the target data to the requester.
In a specific implementation process, please refer to fig. 2, where fig. 2 is a schematic flowchart of an embodiment of the network addressing method based on a distributed system according to the present application, and a user can query data in the distributed file system by using the data query method according to the present embodiment.
S201: a data access request is received from a requestor.
Since all network nodes in the distributed file system are equal and there is no central node, each node has independent functions of receiving and sending information.
The requester may be a network node that receives the user query instruction, or may be another node that has a network connection with the node and forwards the data access request. The data access request includes the targeted data hash value.
Specifically, when the requester is a network node that receives a user query instruction, the requester first performs a hash operation on the target data to obtain a hash value of the target data.
In a specific embodiment, a provider identity hash value for uploading the target data, an uploading tool hash value for uploading the target data, and a data hash value of the target data are associated to calculate a target data hash value corresponding to the target data. In the present embodiment, the above-described association relationship is embodied by a merkel data structure.
The merkel data structure, Merkle DAG, is known collectively as merkel directed acyclic graph, and can use multiple hashes to uniquely identify the content of a piece of data. In the present embodiment, the data hash value of the target data is constructed by this characteristic.
As shown in fig. 3. The data key is a data hash value, the Identification is an identity information hash value, the FileHash is a data hash value of the target data, the AppID is an uploading tool hash value, and the UserID is a provider identity hash value. And the initiator performs hash processing on the uploading tool hash value AppID and the provider identity hash value UserID, for example, the identity information hash value Identification of the target data is obtained through XOR processing, and then performs hash processing on the data hash value FileHash and the identity information hash value Identification of the target data to obtain the data hash value Datakey of the target data. The target data is uniquely identified by a data hash value formed by multiple hashing.
When data transmission is performed in a peer-to-peer network, a method generally adopted for checking data integrity is to divide a large file into small data blocks. Therefore, the hash value FileHash of the target data in the present embodiment is obtained by hashing hash values of a plurality of data blocks constituting the limited data, that is, is formed by the principle of the merckel tree. In particular, the data is divided into small data blocks, with a corresponding hash value corresponding to the data block. And further going upwards, not directly calculating the hash value of each data block, but performing hash combination on the hash values of two adjacent data blocks at the same level to obtain a sub-hash, and repeating the steps to perform hash combination on the two adjacent hash values at the same level at the previous level to obtain a sub-hash at a higher level, and repeating the process to obtain the data hash value of the final target data.
As shown in fig. 4, fig. 4 is a flowchart illustrating an embodiment of a process for forming a hash value of target data. The target data are divided into 4 data blocks including L1, L2, L3 and L4, and Hash processing is performed on each data block to obtain a Hash value 0-0 of L1, a Hash value Hash0-1 of L2, a Hash value Hash1-0 of L3 and a Hash value Hash1-1 of L4. The Hash value 0-0 and the Hash value 0-1 are subjected to primary Hash calculation to obtain a Hash value Hash0, the Hash value 1-0 and the Hash value Hash1-1 are subjected to primary Hash calculation to obtain a Hash value Hash1, and the Hash value Hash0 and the Hash value Hash1 are subjected to primary Hash calculation to obtain a data Hash value FileHash of the target data.
And after the requester obtains the target data hash value, judging whether the node comprises a data hash value matched with the target data hash value, if not, generating the data access request, and sending the data access request to the next network node closest to the requester.
The nearest network node is the network node that is the closest logical distance from the node hash value of the requestor. Specifically, each network node has a unique node number, the node numbers of all network nodes in the distributed file system are subjected to hash processing through a uniform hash coding rule to obtain a node hash value of each network node, a difference value is obtained according to a logical difference value between the node hash values, for example, through exclusive or operation, and a routing or traversal sequence table among the plurality of network nodes is determined. For example, a binary tree form routing sequence table is generated and stored in the network node, and the network node can know the next node nearest to the node through the routing sequence table. Optionally, the routing sequence table may also be stored to a distributed hash table of the network node.
Each network node includes a distributed hash table in which the node hash value of the network node and the data hash value stored by the network node are stored. The DHT arrays of all network nodes form a DHT of the distributed system. The distance between the data hash value stored on the network node and the node hash value of the node is the minimum. When the target data is queried, the target data hash value is queried from the network node with the smallest target data hash value normally.
In an optional manner, since the memory space is limited, the general data information itself is not stored in the distributed hash table of the node, so that the distributed hash table further includes an actual storage address of the data information, and when accessing, the data information can be obtained by calling the storage address.
As shown in table 1 below, table 1 is a distributed hash table example table.
TABLE 1
The node information NodeID _1 is a node hash value, and the content key value DataKey is a hash value of the data information.
In order to improve the efficiency of data access and save the memory space of the network nodes, the corresponding relationship of all the network nodes of the distributed file system can be divided into a plurality of mutually independent routing sequence tables. Each network node stores therein only a routing sequence table including the network node. Therefore, the requesting party can pass through a plurality of routing sequence tables and forward the data access request, so that the query speed is greatly improved.
In an optional application scenario, the hash coding rule for the requester to perform hash processing on the target data is the same as the hash coding rule for the network node to generate the node hash value, and the hash coding rule is generated by the principle of the mercker data structure.
After the network node receives the data access request, in order to avoid misoperation caused by network jitter or other reasons, firstly, the target data hash value and the hash value of the node in the distributed hash table of the distributed file system are used for determining whether the node is closest to the requester, and if the node is not closest, the data access request is forwarded to a third party node closer to the requester.
S202: it is determined whether the node can provide the target data to which the data access request is directed.
Optionally, when the network node receives the data access request and determines that the node is closest to the requester, it determines whether a restricted data hash value matching the target data hash value exists in the distributed hash table of the node, and if so, determines that the target data is restricted data and cannot be provided.
Wherein the limited data is non-transmittable data information. The forming process of the limited data hash value is the same as the forming process and the rule of the target data hash value, the forming process is embodied by a Mercker data structure, and the provider identity hash value of the uploaded limited data, the uploading tool hash value of the uploaded limited data and the data hash value of the limited data are associated to calculate the limited data hash value corresponding to the limited data. Specifically, the restricted data hash value is obtained by performing hash processing on the identity information hash value of the restricted data and the data hash value of the restricted data; the identity information hash value is obtained by carrying out hash processing on the provider identity hash value of the uploaded limited data and the upload tool hash value; the data hash value of the limited data is obtained by hashing hash values of a plurality of data blocks constituting the limited data.
The restricted data hash value may be stored in a distributed hash table of the distributed file system, for example, the hash value of the data information is set and labeled, for example, the hash value is calibrated by adding a Block _ flag. Or in a restricted data table other than the DHT. As shown in table 2, table 2 is a table of examples of restricted data.
TABLE 2
| Restricted data table |
| Hash(DataKey_1) |
| Hash(DataKey_2) |
| …… |
| Hash(DataKey_n) |
All the restricted data hash values in the restricted data table and the restricted data hash values in the DHT are inaccessible.
In a specific scenario, when the node receives the data access request, it is determined whether a restricted data hash value matching the target data hash value exists in a restricted data table in the distributed system, and if so, it is determined that the target data is restricted data and cannot be provided. And if the limited table does not comprise the limited data hash value equal to the target data hash value, further judging whether a limited data hash value matched with the target data hash value exists in the distributed hash table, and if so, determining that the target data cannot be provided. Any of the above embodiments can be implemented by determining whether the node stores a restricted data hash value equal to the target data hash value.
And if the limited data hash value matched with the target data hash value does not exist in the data limited table and the distributed hash table of the distributed system, determining that the target data can be provided.
S203: if the target data is determined to be unavailable, the target data is not returned to the requester.
Specifically, if it is determined that the target data is not available, a prompt message indicating that the search is unsuccessful is returned to the network node such as the requester receiving the query instruction or the absence of the message is displayed, so as to prevent the limited data from being propagated.
In another embodiment, in order to further reduce the probability of success of querying restricted data, when the target data is determined to be unavailable, the target data is not only not returned to the requester, but also the hash value of the target data is deleted from the DHT or the restricted data table of the DHT.
If the target data is available data, whether the target data exists in the node is further determined, and if the target data exists, the target data is returned to the requester. If the target data does not exist in all the network nodes, returning prompt information of query failure, which is not limited herein.
Different from the prior art, the network node of the embodiment receives a data access request from a requester, determines whether the node can provide target data pointed by the data access request, and does not return the target data to the requester when determining that the target data cannot be provided. Through the judgment of the target data access authority, the files of the improper contents can be effectively prevented from being diffused, and the adverse social influence caused by the improper contents is reduced.
In order to provide a query judgment basis for the data query method, the limited data hash value is edited in advance in the present application, in the above embodiment, the limited data value can be edited in two ways, one is to set and label the hash value of the data information stored in the distributed hash table of the distributed file system, and the other is to establish the limited data table, and store the index value, such as the hash value, of the data information for representing the limited authority into the limited data table.
If the labeling is carried out by setting and labeling the hash value of the data information stored in the distributed hash table of the distributed file system, the distributed hash table needs to be established first, namely, the hash value of the data in the distributed file system is stored in the corresponding network node first. As shown in fig. 5, fig. 5 is an interaction diagram of a requester and a network node for data storage, where the requester sends a storage request including a target hash value to the network node 1 closest to the requester; the network node 1 value determines the distance between the node and the target data, and returns the distance to the requester; and forwarding the storage request to a network node 2 closest to the network node 1, wherein the network node 2 determines the distance between the node and the target data and sends the distance to a requester. When the requester determines that the target data hash value is closest to the network node 2, the requester sends the target data to the network node 2 for storage. Please refer to fig. 6 for the above specific implementation process.
Fig. 6 is a schematic flowchart of an embodiment of a data storage method based on a distributed file system according to the present application. The method comprises the following steps:
s601: a storage request including a target data hash value is received from the requestor.
Before storing the target data hash value, the distributed file system calculates a node hash value of each network node in the distributed file system according to the same hash coding rule, wherein the node hash value is established in a form of a Mercker data structure. And solving a difference value by performing exclusive-or operation according to the logical difference value between the node hash values, wherein a routing or traversal sequence table among the plurality of network nodes, such as a generated binary tree-form routing sequence table, stores the routing sequence table into the network node, and the network node can know the nearest next node of the node through the distributed hash table.
In order to improve the efficiency of data access and save the memory space of the network nodes, the corresponding relation of all the network nodes can be divided into a plurality of mutually independent routing sequence lists. Each network node stores therein only a routing sequence table including the network node. Therefore, the requesting party can pass through a plurality of routing sequence tables and forward the data access request, so that the query speed is greatly improved.
In order to store the data hash value of the target data in the network node with the node hash value closest to the data hash value, the initiator calculates the target data according to the hash coding rule to obtain the target data hash value. And judging whether the target data hash value is equal to the node hash value of the node, and if so, storing the target data hash value in a distributed hash table of the node. And if the data hash values are not equal, generating a storage request based on the target data hash value, and sequentially forwarding the storage request to all network nodes in the distributed system according to the node sequence recorded in the routing sequence table. And storing the node hash value of the network node and the data hash value stored by the network node into a distributed hash table of the network node. The DHT arrays of all the network nodes form a DHT of the distributed system.
The calculation process of the target data hash value is to establish an association relationship among a provider identity hash value for uploading the target data, an uploading tool hash value for uploading the target data and a data hash value of the target data to calculate a target data hash value corresponding to the target data. Specifically, a provider identity hash value and an uploading tool hash value of the target data to be uploaded and hash values of a plurality of data blocks constituting the target data are obtained; performing hash calculation on the provider identity hash value and the uploading tool hash value to obtain an identity information hash value of the target data; carrying out hash calculation on the hash values of the data blocks to obtain a data hash value of the target data; and performing hash calculation on the identity information hash value and the data hash value at least once to obtain the target data hash value.
The data hash value of the target data is formed by the merkel tree principle. Specifically, the data is first divided into small data blocks, with a corresponding hash value corresponding to the data block. And further going upwards, not directly calculating the hash value of each data block, but performing hash combination on the hash values of two adjacent data blocks at the same level to obtain a sub-hash, repeating the steps, performing hash combination on the two adjacent hash values at the same level at the previous level to obtain a sub-hash at a higher level, and repeating the process to obtain the data hash value of the final target data.
In an optional manner, since the memory space is limited, the general data information itself is not stored in the distributed hash table of the node, so that the distributed hash table further includes an actual storage address of the data information, and when accessing, the data information can be obtained by calling the storage address.
S602: and determining the distance between the hash value of the node and the target data hash value by using the target data hash value and the hash value of the node in the distributed hash table of the distributed file system.
And after receiving the storage request, the network node calculates the distance between the target data hash value and the node hash value of the node.
S603: returning the distance to the requester to enable the requester to determine that the network node with the minimum distance to the target data hash value stores the target data hash value.
The network node returns feedback information including the distance and the node hash value to the requester, the requester finds the network node with the minimum distance from the target data hash value after receiving the feedback information of all network card nodes in the distributed network node, and sends request information for storing the target data hash value to the network node, and the corresponding network node stores the target data hash value.
In an alternative embodiment, if the data size of the target data itself is not large, such as less than 1kb, the target data and the target data hash value may be stored directly in the DHT of the network node. If the data size of the target data is larger than a set value, such as 1kb, in order to save the space of the network node, the target data is stored in other databases, the actual storage address of the data information is stored in the distributed hash table, and the data information can be obtained by calling the storage address during access.
It should be noted that each network node may store multiple target data hash values, which is not limited herein. If the node hash value of the network node is 200, the first target data hash value is 199, the second target data hash value is 201, the distance between the node hash value of the network node and the first target data hash value is the minimum value 1, and the first target data hash value and the second target data hash value are both stored in the distributed hash table corresponding to the network node.
If the distance between the target data hash value and the node hash values of the two network nodes is equal, if the target data hash value is 100, the node hash value of the first network node is 99, the node hash value of the second network node is 101, and the distance between the node hash values of the two network nodes and the target data hash value is 1, the storage nodes may be randomly allocated, may also be allocated according to the query sequence, may also be both stored by the two network nodes, and is not limited herein.
If the authority setting is performed by storing the index value, such as the hash value, used for representing the limited authority by the data information into the limited data table, the limited data can also be stored into the network node with the node hash value closest to the hash value of the limited data by the same storage method.
In any of the above embodiments, the data editing method shown in fig. 7 below can be used to edit the authority of the data information.
S701: an edit request for a target data right is received from a requestor.
S702: determining whether the requestor is an owner or a handler of the target data.
In order to protect the authority of the target data owner as much as possible and avoid malicious tampering of the network information, only the data owner or the data disposables can modify or perform other editing actions on the data. The network node according to this embodiment further determines whether or not the requester is the owner of the target data after receiving the edit request.
S703: and if the target data authority is determined to be the owner or the disposable party, the editing request is responded to edit the target data authority, so that whether data can be provided or not is operated according to the edited target data authority when an access request for the target data is subsequently received.
Specifically, after receiving the edit request, the network node determines whether the data is limited data before editing the authority.
And if the data is limited data, determining the editing mode of the limited data.
In an alternative embodiment, the index value of the restricted data in the DHT for expressing the access right is set as the restricted data index value. Such as by adding a setting callout such as in the form of a Block Flag.
In another alternative embodiment, the restricted data hash value of the restricted data may also be stored in a restricted data table independent of the distributed hash table of the distributed file system by rebuilding the restricted data table, and storing the restricted data table in each network node.
And after the target data is determined to be the limited data, carrying out primary hash processing on the index value of the limited data to obtain a limited data hash value, and storing the limited data hash value of the target data. Further, in order to prevent the limited authority of the target data from being broken, in another optional implementation manner, the limited data hash value may be subjected to at least one hash processing again, and the target data hash value after being processed for multiple times is stored in the corresponding distributed hash table or the limited data table.
When the restricted data hash value is stored, the restricted data hash value is stored.
After the authority editing of the limited data hash value is finished, when a user or a request party carries out data addressing or downloading subsequently, whether the limited data hash value matched with the target data hash value exists in the distributed hash table or the limited data table can be judged directly through comparison with the limited data hash value, if the limited data hash value matched with the target data hash value exists, the target data cannot be provided, and the target data is not returned to the request party.
Different from the prior art, the embodiment edits the access right of the target data, so that when the target data is subsequently queried, the judgment of the access right of the target data can be determined based on the edited result, the file of the improper content can be effectively prevented from being diffused, and the adverse social influence caused by the improper content is reduced.
In terms of hardware architecture, the present application proposes a data query apparatus based on a distributed file system, and referring to fig. 8, the data query apparatus 800 of this embodiment includes a memory 81 and a processor 82 coupled via a bus, and a communication circuit 83. The memory 81 stores a computer program, which can be run on the processor 82, and the processor 82 can implement any of the above-described network addressing methods when executing the computer program.
The data query apparatus 800 is a network node, and the network node includes a computer or other devices connected to a network having an independent address and a function of transmitting or receiving data. Such as workstations, clients, network users, personal computers, servers, printers, or other network-connected devices, etc., the device having its own unique network address may become a network node.
The memory 81 is used for storing the distributed hash table, the routing sequence table and the restricted data hash table of the node. The distributed hash table of the network node comprises a node hash value of the network node and a data hash value of data stored by the network node. The DHT arrays of all the network nodes form a DHT of the distributed system. The network node can know the next node nearest to the node through the routing sequence table.
The communication circuit 83 is used to receive a data access request from a requester. The processor 82 determines whether the node can provide the target data to which the data access request is directed. If the target data is determined to be unavailable, the target data is not returned to the requester.
When the processor 82 receives the data access request and determines that the node is closest to the requester, it determines whether a restricted data hash value matching the target data hash value exists in the distributed hash table of the node, and if so, determines that the target data cannot be provided.
The restricted data hash value is a hash value corresponding to the non-transmittable data information. The restricted data hash value may be stored in a distributed hash table of the distributed file system, for example, by setting and marking the hash value of the data information for which the authority is set, the restricted data hash value may also be stored in a restricted data table other than the distributed hash table.
All the restricted data hash values in the restricted data table and the restricted data hash values in the DHT are inaccessible.
And if the limited data hash value matched with the target data hash value does not exist in the data limited table and the distributed hash table of the distributed system, determining that the target data can be provided.
If the processor 82 determines that the target data is not available, the communication circuit 83 returns a prompt indicating that the search is not successful to the network node such as the requester receiving the query instruction or displays that the information does not exist, so as to prevent the limited data from being propagated.
Unlike the prior art, the processor of this embodiment receives a data access request from a requester, determines whether the node can provide target data pointed by the data access request, and does not return the target data to the requester when determining that the target data cannot be provided. Through the judgment of the target data access authority, the files of the improper contents can be effectively prevented from being diffused, and the adverse social influence caused by the improper contents is reduced.
In order to provide a query judgment basis for the data query method, the limited data hash value is edited in advance in the present application, and correspondingly, in terms of a hardware architecture, a data editing apparatus based on a distributed file system is proposed in the present application, referring to fig. 9, a data editing apparatus 900 of the present embodiment includes a memory 91, a processor 92 and a communication circuit 93 coupled through a bus. The memory 91 stores a computer program that can be executed on the processor 92, and the processor 92 can implement the data editing method according to any one of the above embodiments when executing the computer program.
The data editing apparatus 900 is a network node, and the network node includes a computer or other devices connected to a network having an independent address and a function of transmitting or receiving data. Such as workstations, clients, network users, personal computers, servers, printers, or other network-connected devices, etc., the device having its own unique network address may become a network node.
The communication circuit 93 has an edit request for receiving a target data right from a requester.
In an alternative embodiment, when storing the target data, the processor 92 adds the identity information of the network node that uploaded the target data to the distributed hash table of the network node at the same time, so as to facilitate the identity judgment.
The processor 92 is further configured to, if the owner is determined, edit the target data permission in response to the edit request, so that when an access request for the target data is subsequently received, an operation is performed on whether data can be provided according to the edited target data permission.
Specifically, after the communication circuit 93 receives the edit request, the processor 92 further determines whether the data is restricted data, sets the target data hash value in the distributed hash table as a restricted data hash value, for example, by adding a setting Flag such as a Block Flag to perform a Flag, or by reestablishing a restricted data table independent of the distributed hash table of the distributed file system, and stores the target data hash value in the restricted data table, where all data hash values in the restricted data table cannot be provided.
After the authority editing of the restricted data hash value is completed, when a user or a requester subsequently queries or downloads data, the processor 92 may directly compare the restricted data hash value with the restricted data hash value to determine whether a restricted data hash value matching the target data hash value exists in the distributed hash table or the restricted data table, and if the restricted data hash value matching the target data hash value exists, determine that the target data cannot be provided, and do not return the target data to the requester.
Different from the prior art, the processor in the embodiment edits the access right of the target data, so that when the target data is subsequently queried, the judgment of the access right of the target data can be determined based on the edited result, the file of the improper content can be effectively prevented from being diffused, and the adverse social influence caused by the improper content is reduced.
In the aspect of a computer program, if it is sold or used as a stand-alone software product, it may be stored in a computer-readable storage medium, and thus the present application proposes a computer-readable storage medium. Referring to fig. 10, fig. 10 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application, in which a computer program 1001 is stored in the computer-readable storage medium 1000 of the present embodiment, and the computer program is executed by a processor to implement the data query method or the data compiling method.
The computer-readable storage medium 1000 may be a medium that can store a computer program, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or may be a server that stores the computer program, and the server can send the stored computer program to another device for running or can run the stored computer program by itself. The computer readable storage medium 1000 may be a combination of multiple entities from a physical point of view, for example, multiple servers and multiple memories, or multiple combinations of memories and multiple removable disks.
The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (13)
1. A network addressing method based on a distributed file system is characterized by comprising the following steps:
receiving a data access request from a requester; the data access request comprises a data access request carrying a target data hash value;
determining whether the node can provide target data pointed by the data access request; the method specifically comprises the following steps: determining whether a restricted data hash value matching the target data hash value exists in a distributed hash table of the distributed file system; or determining whether a limited data hash value matched with the target data hash value exists in a limited data table outside a distributed hash table of the distributed file system; if the current time slot exists, determining that the current time slot is unavailable;
if the target data is determined to be unavailable, not returning the target data to the requester;
the distributed hash table comprises a node number and a data hash value stored in the node, and the limited data hash value in the distributed hash table is marked by a set mark; the limited data table at least comprises a limited data hash value, wherein the limited data hash value is obtained by carrying out hash processing on the identity information hash value of the limited data and the data hash value of the limited data; the identity information hash value is obtained by carrying out hash processing on the provider identity hash value and the uploading tool hash value of the uploaded limited data; the data hash value of the limited data is obtained by hashing hash values of a plurality of data blocks constituting the limited data.
2. The network addressing method of claim 1, wherein the step of not returning the target data to the requestor if it is determined that the target data is not available comprises:
and if the target data is determined to be unavailable, not returning the target data to the requester, and deleting the hash value of the target data from the DHT or the restricted data table of the DHT.
3. The network addressing method of claim 1,
the step of receiving a data access request from a requestor comprises: receiving a data access request carrying a target data hash value from a requester;
the step of receiving a data access request from a requestor thereafter comprises:
determining whether the node is closest to the requester by using the hash value of the target data and the hash value of the node in the distributed hash table of the distributed file system;
and if the data access request is not the latest, forwarding the data access request to a third-party node closer to the requester.
4. The network addressing method of claim 3, wherein the step of forwarding the data access request to a third party node closer to the requestor comprises:
and forwarding the data access request to a plurality of third-party nodes in parallel.
5. The network addressing method of claim 1, further comprising:
if the target data can be provided, determining whether the target data pointed by the data access request exists in the node;
and if the target data is determined to exist, returning the target data to the requester.
6. A data editing method based on a distributed file system is characterized by comprising the following steps:
acquiring provider identity hash values and uploading tool hash values of uploaded target data and hash values of a plurality of data blocks forming the target data;
performing hash calculation on the provider identity hash value and the uploading tool hash value to obtain an identity information hash value of the target data; carrying out hash calculation on the hash values of the data blocks to obtain a data hash value of the target data;
performing hash calculation on the identity information hash value and the data hash value at least once to obtain the target data hash value;
receiving an editing request of target data hash value authority from a requester; judging whether the target data is limited data or not based on the editing request;
determining whether the requestor is an owner or a handler of the target data;
if the target data hash value authority is determined to be the owner or the disposable party, the editing request is responded to edit the target data hash value authority, so that when an access request to the target data is received subsequently, whether data can be provided or not is operated according to the edited target data hash value authority; the method specifically comprises the following steps: if the target data is limited data, setting the target data hash value in a distributed hash table of the distributed file system as a limited data hash value; or forwarding the target data hash value to a restricted data table outside the distributed hash table of the distributed file system for storage.
7. The data editing method according to claim 6, wherein the step of receiving an editing request of the target data hash value authority from the requester is preceded by:
receiving a storage request including a target data hash value from the requestor;
determining the distance between the node and the target data by using the hash value of the target data and the hash value of the node in the distributed hash table of the distributed file system;
returning the distance to the requester so that the requester determines that the network node with the minimum distance between the node and the target data stores the target data hash value; or forwarding the storage request to the network node with the minimum distance between the node and the target data so as to store the target data hash value.
8. The data editing method of claim 6, wherein determining that the target data is restricted further comprises:
and storing the restricted data hash value of the target data, or performing hash processing on the restricted data hash value at least once again, and storing the hash value obtained after the hash processing.
9. The data editing method of claim 8, wherein the step of storing the restricted data hash value of the target data comprises:
determining whether the node is the node closest to the limited data hash value or not by using the limited data hash value of the target data and the hash value of the node in the distributed hash table;
and if not, forwarding the limited data hash value to a third party node closer to the limited data hash value.
10. A data querying device based on a distributed file system, the data querying device comprising a communication circuit, a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.
11. The data query device of claim 10, wherein the distributed file system is any one of an interplanetary file system, a FastDFS distributed system, a GFS distributed file system, a TFS distributed file system, and a HDFS distributed file system.
12. A data editing apparatus based on a distributed file system, the data editing apparatus comprising a communication circuit, a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 6 to 9 when executing the computer program.
13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any of claims 1-5 or 6-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810892073.4A CN109246190B (en) | 2018-08-07 | 2018-08-07 | Network addressing method, data editing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810892073.4A CN109246190B (en) | 2018-08-07 | 2018-08-07 | Network addressing method, data editing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109246190A CN109246190A (en) | 2019-01-18 |
| CN109246190B true CN109246190B (en) | 2021-06-01 |
Family
ID=65069961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810892073.4A Active CN109246190B (en) | 2018-08-07 | 2018-08-07 | Network addressing method, data editing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109246190B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110009759A (en) * | 2019-04-18 | 2019-07-12 | 众安信息技术服务有限公司 | Work attendance method, attendance checking system and Work attendance device based on block chain |
| CN110290182B (en) * | 2019-05-31 | 2021-01-26 | 北京邮电大学 | Distributed Web application storage system, service system and access method |
| CN112181242B (en) * | 2019-07-02 | 2021-11-02 | 北京百度网讯科技有限公司 | Page display method and device |
| CN111092909A (en) * | 2019-12-30 | 2020-05-01 | 深圳市优权天成科技有限公司 | Decentralized remote control downloading algorithm and equipment |
| CN111984322B (en) * | 2020-09-07 | 2023-03-24 | 北京航天数据股份有限公司 | Control instruction transmission method and device |
| CN116107520B (en) * | 2023-04-13 | 2023-06-30 | 北京中科特瑞科技有限公司 | S3 object storage protocol encrypted data storage method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101753610A (en) * | 2008-12-19 | 2010-06-23 | 华为技术有限公司 | Distributed network structure method, device, system and task processing method |
| CN102984020A (en) * | 2011-09-07 | 2013-03-20 | 天讯天网(福建)网络科技有限公司 | Cloud-computation-based net surfing monitoring method and monitoring system and monitoring server |
| CN107172102A (en) * | 2017-07-13 | 2017-09-15 | 上海联影医疗科技有限公司 | Data access method, system and storage medium |
| CN108009646A (en) * | 2017-11-30 | 2018-05-08 | 深圳市轱辘车联数据技术有限公司 | Vehicle data processing method and server |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104506889A (en) * | 2014-12-30 | 2015-04-08 | 青岛海信电器股份有限公司 | Method, terminal, servers and system for playing video |
| US10936548B2 (en) * | 2016-03-15 | 2021-03-02 | Microsoft Technology Licensing, Llc | File synchronization pausing for individual files |
| CN106100902B (en) * | 2016-08-04 | 2020-04-03 | 腾讯科技（深圳）有限公司 | Cloud index monitoring method and device |
| CN107992491A (en) * | 2016-10-26 | 2018-05-04 | ***通信有限公司研究院 | A kind of method and device of distributed file system, data access and data storage |
-
2018
- 2018-08-07 CN CN201810892073.4A patent/CN109246190B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101753610A (en) * | 2008-12-19 | 2010-06-23 | 华为技术有限公司 | Distributed network structure method, device, system and task processing method |
| CN102984020A (en) * | 2011-09-07 | 2013-03-20 | 天讯天网(福建)网络科技有限公司 | Cloud-computation-based net surfing monitoring method and monitoring system and monitoring server |
| CN107172102A (en) * | 2017-07-13 | 2017-09-15 | 上海联影医疗科技有限公司 | Data access method, system and storage medium |
| CN108009646A (en) * | 2017-11-30 | 2018-05-08 | 深圳市轱辘车联数据技术有限公司 | Vehicle data processing method and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109246190A (en) | 2019-01-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109246190B (en) | Network addressing method, data editing method, device and storage medium | |
| CN110958117B (en) | Block chain interoperability with support for zero knowledge proof | |
| US9894168B2 (en) | Locality based content distribution | |
| CN109831487B (en) | Fragmented file verification method and terminal equipment | |
| TWI727467B (en) | Trustworthiness verification method, system, device and equipment of alliance chain | |
| US7478120B1 (en) | System and method for providing a peer indexing service | |
| US20200007558A1 (en) | Information verification system, information verification device, method and program | |
| CN111630830A (en) | Distributed blockchain data storage under account model | |
| US20210158353A1 (en) | Methods, systems, apparatuses, and devices for processing request in consortium blockchain | |
| TWI712972B (en) | Trustworthiness verification method, system, device and equipment of alliance chain | |
| CN111226206B (en) | Taking a snapshot of blockchain data | |
| WO2020197514A1 (en) | A distributed hash table based blockchain architecture for resource constrained environments | |
| CN112699094B (en) | File storage method, data retrieval method, corresponding device and system | |
| CN110597922A (en) | Data processing method, device, terminal and storage medium | |
| CN110910143A (en) | Identity identification generation method, device, related node and medium | |
| WO2023073103A1 (en) | Methods and systems for distributed blockchain functionalities | |
| CN111447265A (en) | File storage method, file downloading method, file processing method and related components | |
| CN111316256A (en) | Taking snapshots of blockchain data | |
| CN113542405A (en) | Block chain-based network communication system, method, device and storage medium | |
| US20230325833A1 (en) | Blockchain-based data processing method and apparatus, device, storage medium, and program product | |
| Matsuoka et al. | Blockchain and DHT Based Lookup System Aiming for Alternative DNS | |
| CN109657801B (en) | Shunting method and device of recommendation system and readable storage medium | |
| US20220092104A1 (en) | System for automatic management and depositing of documents (images) hash in block-chain technology | |
| Ji et al. | A personal information management using blockchain | |
| CN109032804B (en) | Data processing method and device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |