CN118101680A - Shared file distributed storage method and related equipment - Google Patents
Shared file distributed storage method and related equipment Download PDFInfo
- Publication number
- CN118101680A CN118101680A CN202410230100.7A CN202410230100A CN118101680A CN 118101680 A CN118101680 A CN 118101680A CN 202410230100 A CN202410230100 A CN 202410230100A CN 118101680 A CN118101680 A CN 118101680A
- Authority
- CN
- China
- Prior art keywords
- storage
- file
- shared file
- shared
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 14
- 230000005540 biological transmission Effects 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012986 modification Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a shared file distributed storage method and related equipment. In the system, the shared file sets are stored in a distributed manner through the storage nodes and the terminal nodes, so that each terminal node contains a part of redundant backup aiming at the shared file, when the storage node fails, a new storage node can be obtained based on the redundant backup stored by the terminal node to continue file sharing, the operation of the file sharing system is not influenced by the failure of a single storage node, and the operation stability of the file sharing system is improved. The shared file distributed storage method and the related equipment provided by the application can be used in the field of distributed systems or the field of finance. The foregoing is merely exemplary, and is not intended to limit the application field of the shared file distributed storage system provided by the present application.
Description
Technical Field
The present application relates to the field of distributed systems, and in particular, to a method and related device for storing a shared file in a distributed manner.
Background
The file sharing system is a software or platform which is convenient for sharing, storing and managing files, and is widely applied to team cooperation to improve the team cooperation efficiency. Users can upload and store various types of files through the file sharing system so as to share file information to other users, and each user can cooperatively edit and modify the same file in the sharing system so as to ensure the real-time updating of the shared file.
The current commonly used file sharing system is a centralized file sharing system, and the centralized file sharing system is often used for centrally managing shared files based on servers corresponding to single or few storage nodes, so that a plurality of users can access and edit the same file at the same time, and the collaborative work efficiency of a team is promoted. However, since centralized management of shared files depends on a single or a small number of storage nodes, failure of the storage nodes directly results in failure of the file sharing system, and the running stability of the file sharing system is poor.
Therefore, how to solve the problem of poor running stability of the file sharing system in the prior art is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
Based on the above problems, in order to solve the problem of poor running stability of a file sharing system in the prior art, the application provides a shared file distributed storage method and related equipment.
The embodiment of the application discloses the following technical scheme:
In a first aspect, the application discloses a shared file distributed storage system, which comprises a storage node, a plurality of terminal nodes, a fault processing module and a file transmission module;
the storage node is used for storing the shared file set; the shared file set comprises a plurality of shared files under different modified versions;
The terminal node is used for carrying out distributed storage on the shared file set according to preset storage redundancy; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot;
the fault processing module is used for determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node;
The file transmission module is used for transmitting the redundant backup of the shared file to the new storage node based on the shared file set and the backup storage data of the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
Optionally, the system further comprises: a file synchronization module;
The file synchronization module is used for sending a shared file synchronization signal to the storage node and the plurality of terminal nodes when receiving the updated shared file; the updated shared file is obtained by modifying the shared file through the terminal node based on a user; the shared file synchronization signal includes a version identification and a file snapshot corresponding to the updated shared file.
Optionally, the storage node includes: a file sharing module; the file sharing module is specifically configured to:
determining the version identification and the file snapshot corresponding to the updated shared file according to the shared file synchronization signal;
and generating a file copy corresponding to the updated shared file according to the version identification and the file snapshot corresponding to the updated shared file, and storing the file copy into the shared file set.
Optionally, the storage node further includes: a redundancy detection module;
The redundancy detection module is used for detecting redundancy of a plurality of shared files in the shared file set.
Optionally, the redundancy detection module is specifically configured to:
Acquiring storage redundancy corresponding to each of the plurality of shared files;
Determining the shared file with the storage redundancy lower than a preset threshold value as a first shared file; the first shared file is a file which needs to be subjected to redundancy backup;
and generating a redundant backup of the first shared file, and sending the redundant backup of the first shared file to the plurality of terminal nodes.
Optionally, the fault handling module is specifically configured to:
and when the storage node fails, determining the terminal node with the backup storage data higher than a preset first threshold value and the residual storage capacity of the node higher than a preset second threshold value as a new storage node.
In a second aspect, the application discloses a distributed storage method of shared files, which is applied to a file sharing system comprising a terminal node and a storage node, wherein the storage node is used for storing a shared file set; the shared file set comprises a plurality of shared files under different modified versions; the method comprises the following steps:
According to the preset storage redundancy, carrying out distributed storage on the shared file set; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot;
When the storage node fails, determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node;
transmitting a redundant backup of the shared file to the new storage node based on the shared file set and the backup storage data of the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
Optionally, after the distributed storage of the shared file set according to the preset storage redundancy, the method further includes:
and detecting redundancy of a plurality of the shared files in the shared file set.
In a third aspect, the present application discloses an electronic device, the device comprising: a processor, memory, system bus;
the processor and the memory are connected through the system bus;
The memory is configured to store one or more programs, the one or more programs comprising instructions, which when executed by the processor, cause the processor to perform the shared file distributed storage method.
In a fourth aspect, the present application discloses a computer readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the shared file distributed storage method.
Compared with the prior art, the application has the following beneficial effects: the application provides a shared file distributed storage method and related equipment. The storage node is used for storing a shared file set, and the shared file set comprises a plurality of shared files under different modified versions. The terminal node is used for carrying out distributed storage on the shared file set according to preset storage redundancy. The terminal node stores a plurality of redundant backups of the shared files; the redundant backup includes: version identification and file snapshot; the fault processing module is used for determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node; and the file transmission module is used for transmitting the redundant backup of the shared file to the new storage node based on the backup storage data of the new storage node. In the system, through the distributed storage of the storage nodes and the plurality of terminal nodes to the shared file set, each terminal node contains a part of redundant backup aiming at the shared file, the redundant backup stored by all the terminal nodes comprises the redundant backup of all the shared files in the shared file set, when the storage nodes for storing the shared file set in the shared file system fail, based on the redundant backup stored by the terminal nodes and the storage condition of the terminal nodes to the redundant backup, a new storage node can be obtained to continue file sharing, and the operation of the file sharing system is not influenced by the failure of a single storage node, so that the operation stability of the file sharing system is improved.
Drawings
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.
FIG. 1 is a schematic diagram of a shared file distributed storage system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of another shared file distributed storage system according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a method for distributed storage of shared files according to an embodiment of the present application;
Fig. 4 is a schematic structural diagram of a shared file distributed storage electronic device according to an embodiment of the present application.
Detailed Description
As described above, the file sharing system is a software or platform that facilitates sharing, storing and managing files, and is widely used in team cooperation to improve team cooperation efficiency. Users can upload and store various types of files through the file sharing system so as to share file information to other users, and each user can cooperatively edit and modify the same file in the sharing system so as to ensure the real-time updating of the shared file.
The current commonly used file sharing system is a centralized file sharing system, and the centralized file sharing system is often used for centrally managing shared files based on servers corresponding to single or few storage nodes, so that a plurality of users can access and edit the same file at the same time, and the collaborative work efficiency of a team is promoted. However, since centralized management of shared files depends on a single or a small number of storage nodes, failure of the storage nodes directly results in failure of the file sharing system, and the running stability of the file sharing system is poor.
Therefore, how to solve the problem of poor running stability of the file sharing system in the prior art is a technical problem that needs to be solved by those skilled in the art.
In order to solve the above problems, the present application provides a method and related device for distributed storage of a shared file, where the shared file distributed storage system provided by the present application includes a storage node, a plurality of terminal nodes, a failure processing module, and a file transmission module. The storage node is used for storing a shared file set, and the shared file set comprises a plurality of shared files under different modified versions. The terminal node is used for carrying out distributed storage on the shared file set according to preset storage redundancy. The terminal node stores a plurality of redundant backups of the shared files; the redundant backup includes: version identification and file snapshot; the fault processing module is used for determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node; and the file transmission module is used for transmitting the redundant backup of the shared file to the new storage node based on the backup storage data of the new storage node. In the system, through the distributed storage of the storage nodes and the plurality of terminal nodes to the shared file set, each terminal node contains a part of redundant backup aiming at the shared file, the redundant backup stored by all the terminal nodes comprises the redundant backup of all the shared files in the shared file set, when the storage nodes for storing the shared file set in the shared file system fail, based on the redundant backup stored by the terminal nodes and the storage condition of the terminal nodes to the redundant backup, a new storage node can be obtained to continue file sharing, and the operation of the file sharing system is not influenced by the failure of a single storage node, so that the operation stability of the file sharing system is improved.
The shared file distributed storage method and the related equipment provided by the application can be used in the field of distributed systems or the field of finance. The foregoing is merely exemplary, and is not intended to limit the application field of the shared file distributed storage system provided by the present application.
In order to make the present application better understood by those skilled in the art, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, the schematic structural diagram of a shared file distributed storage system according to an embodiment of the present application specifically includes the following nodes and modules:
The shared file distributed storage system provided by the application comprises a storage node 100, a plurality of terminal nodes 200, a fault processing module 300 and a file transmission module 400;
wherein, the storage node 100 is configured to store a shared file set; the set of shared files includes a plurality of shared files under different modified versions.
A file sharing system is a system that allows multiple users to commonly access and manage files, and provides a centralized storage location for shared files, and users may upload, download, edit, etc. the shared files based on the file sharing system. In the shared file distributed storage system, a shared file set which is mainly used for carrying out file sharing service is stored in a storage node corresponding to the storage system. It can be understood that in an actual application scenario, the sharing of files often accompanies operations such as adding, deleting, modifying, etc. of the shared files by a user, so that a plurality of shared files under different modified versions are stored in the shared file set, each shared file of different versions has an independent corresponding version identifier, and the user can select the shared file under the corresponding modified version according to own needs, thereby improving the problem of multi-modified version conflict caused when frequent modification is performed on the shared file.
The terminal node 200 is configured to perform distributed storage on the shared file set according to a preset storage redundancy; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot;
The shared file distributed storage system comprises a plurality of terminal nodes, wherein the terminal nodes are used for carrying out distributed storage on the shared file set according to the preset storage redundancy. The terminal nodes are stored with redundancy backups corresponding to each shared file, and each shared file generates a plurality of self redundancy backups based on the preset storage redundancy and is distributed and stored in a plurality of terminal nodes.
A further understanding may be obtained with particular reference to the schematic diagram of another shared file distributed storage system disclosed in fig. 2. As shown in the figure, the figure comprises a storage node and a plurality of terminal nodes, all the shared files stored in the storage node are shared file sets, and letters and serial numbers corresponding to different files represent different modified versions of the shared file sets. For example, file 1-A1 represents a shared file of shared file 1 in version A1, and file 1-A2 represents a shared file of shared file 1 in another version A2, with the shared file set being made up of multiple different modified versions of the shared file. The redundant backup of part of the shared files in the shared file set is stored in each terminal node, and repeated redundant backup exists among a plurality of different terminal nodes, so that the storage redundancy of the redundant backup of the shared files can be maintained at a certain level, and when a storage node or a certain terminal node fails, the file recovery can be performed based on the redundant backup repeatedly stored by other terminal nodes, thereby ensuring the normal operation of the file sharing system.
The redundant backup of the shared file comprises a version identifier of the shared file and a corresponding file snapshot, wherein the file snapshot of the shared file is used for representing an image or copy of the shared file at a specific time point and is used for recording the file state, the content and the original data of the file at the specific time point so as to be convenient for playing a role in recovering the file. The file snapshot corresponding to the shared file is mainly used for backing up the shared file, and aims to recover the shared file through the corresponding snapshot when the storage node fails and the shared file cannot be normally acquired, so that normal operation of file sharing service is ensured.
In addition, the file snapshot of the shared file accessed in the application is the corresponding file snapshot under different modification versions, the file snapshots under a plurality of modification versions stored in the terminal node can record a plurality of historical versions corresponding to the shared file, and a user can check the modification generated by the shared file in a certain time period by comparing the file snapshots of different versions, thereby quickly selecting the file snapshot to generate the corresponding shared file, and realizing the version control of a single shared file.
The fault processing module 300 is configured to determine a new storage node according to backup storage data of the plurality of terminal nodes and corresponding node remaining storage capacities when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node;
The fault processing module is used for carrying out fault detection on the storage nodes, and when the storage nodes have faults, a new storage node is determined according to the specific backup storage data of each terminal node and the actual residual storage capacity in the nodes. In an actual application scenario, there is often a difference in the storage capacities of the different end nodes, and therefore, in determining a new storage node from a plurality of end nodes, it is necessary to determine the new storage node based on the types of redundancy backups stored inside the end nodes and the actual remaining storage capacities thereof. Specifically, the terminal node with the backup storage data higher than the preset first threshold and the node residual storage capacity higher than the preset second threshold is determined as a new storage node, so that the storage node with more redundant backup storage categories and still with a certain storage capacity is selected as the new storage node, and the storage capacity of the new storage node and the redundant backup storage data can support the normal operation of the file sharing system after the new storage node is determined.
The file transmission module 400 is configured to transmit, to the new storage node, a redundant backup of the shared file based on the shared file set and the backup storage data of the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
After the terminal node is determined to be a new storage node, the node is the node with the most redundant backup storage, but relative to the shared file set, the node still lacks the redundant backup corresponding to a part of the shared files. Thus, after determining a new storage node, the missing redundant backup needs to be transferred to the new storage node. Specifically, based on the shared file set and the backup storage data of the new storage node, a redundant backup of the new storage node relative to the shared file missing from the shared file set is obtained, and the missing redundant backup is transmitted to the new storage node. The new storage node stores the redundant backup corresponding to all the shared files in the shared file set, and generates the corresponding shared files based on the version identification and the file snapshot in the redundant backup, so that the normal operation of the file sharing system is still ensured under the condition that the storage node fails, and the operation stability of the file sharing system is improved.
As an alternative embodiment, the system further comprises: a file synchronization module;
The file synchronization module is used for sending a shared file synchronization signal to the storage node and the plurality of terminal nodes when receiving the updated shared file; the updated shared file is obtained by modifying the shared file through the terminal node based on a user; the shared file synchronization signal includes a version identification and a file snapshot corresponding to the updated shared file.
And storing redundant backup corresponding to part of the shared files in each terminal node, and when a user of the terminal node wants to modify the shared files in the redundant backup, generating corresponding shared files through the stored redundant backup to modify and update the shared files. In the process of modifying and updating the shared file based on the terminal node, the modification of the file is not synchronized to the shared file set of the storage node in real time, and the storage node can update the shared file only when the modified shared file is submitted to the file synchronization module.
Specifically, when the file synchronization module receives the updated shared file, the file synchronization module sends a shared file synchronization signal to the storage node and a plurality of terminal nodes in the system. The shared file synchronization signal comprises a modified version identifier and a file snapshot corresponding to the updated shared file, and when the system has modified updating of the shared file, the corresponding shared file stored in each node is updated after the user submits the updated shared file. The updated shared file has specific version identification and file snapshot, so that a user can select shared files of different versions according to own requirements and acquire a desired shared file based on the corresponding file snapshot, thereby avoiding the problem of file modification conflict caused by frequent modification of multiple users in a file sharing system.
As another alternative embodiment, the storage node includes: a file sharing module; the file sharing module is specifically configured to:
determining the version identification and the file snapshot corresponding to the updated shared file according to the shared file synchronization signal;
and generating a file copy corresponding to the updated shared file according to the version identification and the file snapshot corresponding to the updated shared file, and storing the file copy into the shared file set.
As can be seen from the above description about the file synchronization module, the shared file synchronization signal includes the version identifier and the file snapshot corresponding to the updated shared file, when there is an update of the shared file, the terminal node to which the shared file synchronization signal belongs will transmit the shared file synchronization signal to each node in the system, and the user can select the file snapshot of the corresponding version according to the own requirement to obtain the shared file under the version. However, in order to ensure the reserve integrity of the shared file set for each version of the shared file, when the storage node receives the file synchronization signal, the shared file under the corresponding version is generated in real time according to the version identifier and the file snapshot in the file synchronization signal, so that the storage integrity of the shared file for each version of the shared file is ensured.
As another alternative embodiment, the system further comprises: a redundancy detection module;
The redundancy detection module is used for detecting redundancy of a plurality of shared files in the shared file set.
In an actual file sharing system, there is a certain fluctuation in the storage redundancy of a shared file in each terminal node, because of the increase, deletion, and the like of the shared file. In order to ensure that the storage redundancy of the shared files in each terminal node is maintained at a higher level, so that file recovery is facilitated to each shared file when the storage node fails, the redundancy detection module can detect the redundancy of the shared files in the shared file set in real time, and transmit the storage redundancy of a certain shared file to a plurality of terminal nodes when the storage redundancy of the shared file is lower than a preset threshold value, so that the storage redundancy of the shared file is improved. Specifically, the process of performing redundancy detection by the redundancy detection module may be implemented by the following three steps:
step one, obtaining storage redundancy corresponding to each of the plurality of shared files;
step two, determining the shared file with the storage redundancy lower than a preset threshold value as a first shared file; the first shared file is a file which needs to be subjected to redundancy backup;
And thirdly, generating redundant backup of the first shared file and sending the redundant backup of the first shared file to the plurality of terminal nodes.
Firstly, the redundancy detection module acquires the storage redundancy corresponding to each shared file in the shared file set in real time, and determines the shared file with the storage redundancy lower than a preset threshold as a first shared file. After the first shared file, namely the shared file needing to be subjected to redundancy backup, is determined, the corresponding redundancy backup is generated, and the redundancy backup is sent to a plurality of terminal nodes in the system, so that each shared file in the shared file set can be ensured to have certain storage redundancy, and when the storage node fails, the shared file set can be quickly recovered, and the normal operation of the file sharing system is ensured.
The embodiment provides a shared file distributed storage system, which comprises a storage node, a plurality of terminal nodes, a fault processing module and a file transmission module. The storage node is used for storing a shared file set, and the shared file set comprises a plurality of shared files under different modified versions. The terminal node is used for carrying out distributed storage on the shared file set according to preset storage redundancy. The terminal node stores a plurality of redundant backups of the shared files; the redundant backup includes: version identification and file snapshot; the fault processing module is used for determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node; and the file transmission module is used for transmitting the redundant backup of the shared file to the new storage node based on the backup storage data of the new storage node. In the system, through the distributed storage of the storage nodes and the plurality of terminal nodes to the shared file set, each terminal node contains a part of redundant backup aiming at the shared file, the redundant backup stored by all the terminal nodes comprises the redundant backup of all the shared files in the shared file set, when the storage nodes for storing the shared file set in the shared file system fail, based on the redundant backup stored by the terminal nodes and the storage condition of the terminal nodes to the redundant backup, a new storage node can be obtained to continue file sharing, and the operation of the file sharing system is not influenced by the failure of a single storage node, so that the operation stability of the file sharing system is improved.
The following describes a method for distributed storage of a shared file according to an embodiment of the present application, where the method for distributed storage of a shared file and the system for distributed storage of a shared file described above may be referred to correspondingly.
Referring to fig. 3, the flow chart of a method for distributed storage of shared files according to an embodiment of the present application specifically includes the following steps:
S101: according to the preset storage redundancy, carrying out distributed storage on the shared file set; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot.
S102: when the storage node fails, determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used to represent the current storage capacity of the end node.
S103: transmitting a redundant backup of the shared file to the new storage node based on the shared file set and the backup storage data of the new storage node; the backup storage data is used for representing the category of the redundant backup stored by the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
Optionally, after the distributed storage of the shared file set according to the preset storage redundancy, the method further includes:
and detecting redundancy of a plurality of the shared files in the shared file set.
Optionally, the performing redundancy detection on a plurality of the shared files in the shared file set includes:
Acquiring storage redundancy corresponding to each of the plurality of shared files;
Determining the shared file with the storage redundancy lower than a preset threshold value as a first shared file; the first shared file is a file which needs to be subjected to redundancy backup;
and generating a redundant backup of the first shared file, and sending the redundant backup of the first shared file to the plurality of terminal nodes.
Optionally, after the distributed storage of the shared file set according to the preset storage redundancy, the method further includes:
if the updated shared file is received, sending a shared file synchronization signal to the storage node and the plurality of terminal nodes; the updated shared file is obtained by modifying the shared file through the terminal node based on a user; the shared file synchronization signal includes a version identification and a file snapshot corresponding to the updated shared file.
Optionally, after the sending the shared file synchronization signal to the storage node and the plurality of terminal nodes, the method further includes:
determining the version identification and the file snapshot corresponding to the updated shared file according to the shared file synchronization signal;
and generating a file copy corresponding to the updated shared file according to the version identification and the file snapshot corresponding to the updated shared file, and storing the file copy into the shared file set.
Referring to fig. 4, the schematic structural diagram of a shared file distributed storage electronic device according to an embodiment of the present application includes:
A memory 11 for storing a computer program;
A processor 12 for implementing the steps of a shared file distributed storage method according to any of the method embodiments described above when executing the computer program.
In this embodiment, the device may be a vehicle-mounted computer, a PC (Personal Computer ), or a terminal device such as a smart phone, a tablet computer, a palm computer, a portable computer, etc.
The device may include a memory 11, a processor 12, and a bus 13.
The memory 11 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the device, such as a hard disk of the device. The memory 11 may also be an external storage device of the device, such as a plug-in hard disk provided on the device, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. in other embodiments. Further, the memory 11 may also include both an internal storage unit of the device and an external storage device. The memory 11 may be used not only for storing application software installed in the device and various types of data, such as program codes for executing a failure prediction method, etc., but also for temporarily storing data that has been output or is to be output. The processor 12 may be a central processing unit (Central Processing Unit, CPU) in some embodiments.
The processor 12 may in some embodiments be a central processing unit (Central Processing unit, CPU), controller, microcontroller, microprocessor or other data processing chip for running program code or processing data stored in the memory 11, such as program code for performing a fault prediction method, etc.
The bus 13 may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, or the like. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 4, but not only one bus or one type of bus.
Further, the device may also include a network interface 14, and the network interface 14 may optionally include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the device and other electronic devices.
Optionally, the device may further comprise a user interface 15, the user interface 15 may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 15 may further comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the device and for displaying a visual user interface.
Fig. 4 shows only a device having components 11-15, it will be understood by those skilled in the art that the configuration shown in fig. 4 is not limiting of the device and may include fewer or more components than shown, or may combine certain components, or a different arrangement of components.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment is mainly described in a different point from other embodiments. In particular, for the method apparatus, the electronic device and the vehicle, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant points are referred to in the description of the method embodiments. The method device, the electronic apparatus and the vehicle described above are only schematic, wherein the units described as separate parts may or may not be physically separated, and the parts as unit prompts may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing is only one specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the technical scope of the present application should be included in the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims (10)
1. The shared file distributed storage system is characterized by comprising a storage node, a plurality of terminal nodes, a fault processing module and a file transmission module;
the storage node is used for storing the shared file set; the shared file set comprises a plurality of shared files under different modified versions;
The terminal node is used for carrying out distributed storage on the shared file set according to preset storage redundancy; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot;
the fault processing module is used for determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity when the storage node fails; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node;
The file transmission module is used for transmitting the redundant backup of the shared file to the new storage node based on the shared file set and the backup storage data of the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
2. The system of claim 1, wherein the system further comprises: a file synchronization module;
The file synchronization module is used for sending a shared file synchronization signal to the storage node and the plurality of terminal nodes when receiving the updated shared file; the updated shared file is obtained by modifying the shared file through the terminal node based on a user; the shared file synchronization signal includes a version identification and a file snapshot corresponding to the updated shared file.
3. The system of claim 2, wherein the storage node comprises: a file sharing module; the file sharing module is specifically configured to:
determining the version identification and the file snapshot corresponding to the updated shared file according to the shared file synchronization signal;
and generating a file copy corresponding to the updated shared file according to the version identification and the file snapshot corresponding to the updated shared file, and storing the file copy into the shared file set.
4. The system of claim 1, wherein the storage node further comprises: a redundancy detection module;
The redundancy detection module is used for detecting redundancy of a plurality of shared files in the shared file set.
5. The system according to claim 4, wherein the redundancy detection module is specifically configured to:
Acquiring storage redundancy corresponding to each of the plurality of shared files;
Determining the shared file with the storage redundancy lower than a preset threshold value as a first shared file; the first shared file is a file which needs to be subjected to redundancy backup;
and generating a redundant backup of the first shared file, and sending the redundant backup of the first shared file to the plurality of terminal nodes.
6. The system according to claim 1, wherein the fault handling module is specifically configured to:
and when the storage node fails, determining the terminal node with the backup storage data higher than a preset first threshold value and the residual storage capacity of the node higher than a preset second threshold value as a new storage node.
7. The shared file distributed storage method is characterized by being applied to a file sharing system comprising a terminal node and a storage node, wherein the storage node is used for storing a shared file set; the shared file set comprises a plurality of shared files under different modified versions; the method comprises the following steps:
According to the preset storage redundancy, carrying out distributed storage on the shared file set; the terminal node stores redundancy backups corresponding to the shared files respectively; the redundant backup includes: version identification and file snapshot;
When the storage node fails, determining a new storage node according to the backup storage data of the plurality of terminal nodes and the corresponding node residual storage capacity; the backup storage data is used for representing the category of the terminal node for storing the redundant backup; the node remaining storage capacity is used for representing the current storage capacity of the terminal node;
transmitting a redundant backup of the shared file to the new storage node based on the shared file set and the backup storage data of the new storage node; and the new storage node stores redundancy backups corresponding to all the shared files in the shared file set.
8. The method of claim 7, wherein after the distributed storage of the shared file set according to the preset storage redundancy, further comprising:
and detecting redundancy of a plurality of the shared files in the shared file set.
9. An electronic device, the device comprising: a processor, memory, system bus;
the processor and the memory are connected through the system bus;
the memory is for storing one or more programs, the one or more programs comprising instructions, which when executed by the processor, cause the processor to perform the shared file distributed storage method of any of claims 7-8.
10. A computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the shared file distributed storage method of any of claims 7-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410230100.7A CN118101680A (en) | 2024-02-29 | 2024-02-29 | Shared file distributed storage method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410230100.7A CN118101680A (en) | 2024-02-29 | 2024-02-29 | Shared file distributed storage method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118101680A true CN118101680A (en) | 2024-05-28 |
Family
ID=91157567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410230100.7A Pending CN118101680A (en) | 2024-02-29 | 2024-02-29 | Shared file distributed storage method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118101680A (en) |
-
2024
- 2024-02-29 CN CN202410230100.7A patent/CN118101680A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20150213100A1 (en) | Data synchronization method and system | |
| CN106708653B (en) | Mixed tax big data security protection method based on erasure code and multiple copies | |
| US8745342B2 (en) | Computer system for controlling backups using wide area network | |
| CN111240892B (en) | Data backup method and device | |
| CN111930489B (en) | Task scheduling method, device, equipment and storage medium | |
| CN110753084A (en) | Uplink data reading method, cache server and computer readable storage medium | |
| KR100922584B1 (en) | Distributed object-sharing system and method thereof | |
| CN112612853A (en) | Data processing method and device based on database cluster and electronic equipment | |
| CN111737227A (en) | Data modification method and system | |
| CN113918437A (en) | User behavior data analysis method and device, computer equipment and storage medium | |
| CN112261114A (en) | Data backup system and method | |
| CN108959548B (en) | Service request processing method and device | |
| CN111488124A (en) | Data updating method and device, electronic equipment and storage medium | |
| CN103714059A (en) | Data updating method and device | |
| CN118101680A (en) | Shared file distributed storage method and related equipment | |
| CN112818204B (en) | Service processing method, device, equipment and storage medium | |
| JP2000035911A (en) | Equalization method for data base | |
| CN115470041A (en) | Data disaster recovery management method and device | |
| CN114996057A (en) | Data backup method and device, electronic equipment and computer readable storage medium | |
| CN108196979B (en) | Data backup method and device | |
| CN108268662B (en) | Social graph generation method based on H5 page, electronic device and storage medium | |
| CN113656208B (en) | Data processing method, device, equipment and storage medium of distributed storage system | |
| CN114281604B (en) | Data recovery method and device, electronic equipment and storage medium | |
| CN116302696A (en) | Archive log generation method of database system, storage medium and computer device | |
| CN113626409B (en) | Test data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication |