CN114610679A - Storage device, data storage method thereof and cloud storage system - Google Patents
Storage device, data storage method thereof and cloud storage system Download PDFInfo
- Publication number
- CN114610679A CN114610679A CN202110298190.XA CN202110298190A CN114610679A CN 114610679 A CN114610679 A CN 114610679A CN 202110298190 A CN202110298190 A CN 202110298190A CN 114610679 A CN114610679 A CN 114610679A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- file
- cloud
- cloud storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses storage equipment and data storage method, a cloud storage system thereof, storage equipment includes: the storage module is used for storing a local data file and a network address index file, and the network address index file points to a corresponding cloud storage address in a cloud storage space; the data transmission module is used for transmitting data between the local device and the cloud storage system, and comprises: uploading the local data files meeting the preset conditions to a cloud storage system, and forming a network address index file with the edited cloud storage address. The storage device expands the storage space through the cloud storage system.
Description
Technical Field
The application relates to the technical field of storage, in particular to a storage device, a data storage method thereof and a cloud storage system.
Background
With the development of science and technology, information systems are shifting from being centered on computing to being centered on data, and mass storage will become a core component of national informatization. At present, information resources are increased at the speed of geometric progression, and the related information not only comprises text and images, but also multimedia information such as voice, video and the like. Therefore, many departments and units start to deploy their own data centers, and realize centralized storage, analysis and sharing of mass data. The data storage mode in the existing data center is a cloud storage technology, and is very suitable for storage of mass information, so that a very convenient solution can be provided for the information center, and centralized storage and sharing of data are realized.
However, the existing data storage mode is too rough, the data storage mode does not effectively manage the local storage and cloud storage of the data, a large amount of equipment investment is consumed, and the data is not effectively managed.
Disclosure of Invention
In view of this, the present application provides a storage device, a data storage method thereof, and a cloud storage system to solve the problem that the storage space of the existing storage device is limited.
The present application provides a storage device comprising: the storage module is used for storing a local data file and a network address index file, and the network address index file points to a corresponding cloud storage address in a cloud storage space; the data transmission module is used for transmitting data between the local device and the cloud storage system, and comprises: uploading the local data files meeting the preset conditions to a cloud storage system, and forming and editing a network address index file of the cloud storage address.
Optionally, the data transmission module is further configured to respond to an operation on the network address index file, and access the corresponding cloud storage address and/or download the data file in the corresponding cloud storage address.
Optionally, the method further includes: and the data analysis module is connected to the storage module and the data transmission module and is used for monitoring the size of the storage space of the storage equipment, and when the monitored residual storage space is smaller than a preset value, the data in the storage space is cleaned according to preset conditions.
Optionally, the data analysis module is further configured to record operation information related to data cleaning in the log system while cleaning data in the storage space.
Optionally, the data analysis module is further configured to generate a unique MD5 value of each data file by using a string transformation algorithm for the data files in the storage device, compare the MD5 value of the data file to be uploaded with the MD5 values of the data files in the cloud storage system, and control the data transmission module to upload the data file to be uploaded to the cloud storage system when the same MD5 value does not exist in the cloud storage system.
Optionally, the data transmission module is configured to establish a data transmission path between the external networked terminal device and the cloud storage system.
Optionally, the storage module includes a buffer area, and is configured to buffer data when the data transmission module performs data transmission.
Optionally, the preset conditions include: at least one of the file storage time exceeding a first threshold, the file size exceeding a second threshold, the time since last being opened exceeding a third threshold, and the device storage capacity being less than a fourth threshold.
Optionally, the data files uploaded by the storage device are stored in the cloud storage system in a centralized or distributed manner; when downloading the data file in the cloud storage address, the data transmission module is used for directly downloading the complete file from the cloud storage system, or performing data splicing on a plurality of data blocks after acquiring the plurality of data blocks to acquire the complete data file.
The application also provides a data storage method of the storage device, which comprises the following steps: judging whether a data file locally stored by the equipment meets a preset condition or not; uploading the data file meeting the preset condition to a cloud storage system, and acquiring a cloud storage address of the data file in a cloud storage space; and locally storing a network address index file corresponding to the uploaded data file in the equipment, wherein the network address index file points to the cloud storage address of the data file.
Optionally, the method further includes: and responding to the operation of the network address index file, and accessing the corresponding cloud storage address and/or downloading the data file in the corresponding cloud storage address.
Optionally, the method further includes: generating a unique MD5 value of each data file by a character string transformation algorithm for the data files in the storage device, comparing the MD5 value of the data file to be uploaded with the MD5 value of each data file in the cloud storage system, and uploading the data file to be uploaded to the cloud storage system when the same MD5 value does not exist in the cloud storage system.
Optionally, the method further includes: monitoring the size of the storage space of the storage device, and when the monitored residual storage space is smaller than a preset value, cleaning data in the storage space according to preset conditions.
Optionally, the data analysis module is further configured to record operation information related to data cleaning in the log system while cleaning data in the storage space.
The present application further provides a storage device, comprising: a processor; memory storing a computer program executable by the processor, the computer program when executed being capable of performing a data storage method as claimed in any one of the preceding claims.
The present application further provides a cloud storage system, including: the cloud storage space is used for storing the data file uploaded by the storage device; and the cloud control module is connected with the cloud storage space and used for establishing data transmission connection with the storage equipment, distributing a corresponding cloud storage address for the data file uploaded by the storage equipment, and feeding the cloud storage address back to the storage equipment end.
According to the storage device, the data file is uploaded to the cloud storage system and is accessed through the index file, and the problem that the storage space of the existing storage device is limited is solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a storage device according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of a memory module of a memory device according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a memory module of a memory device according to an embodiment of the present application;
fig. 4 is a schematic diagram illustrating a structure of a cloud storage system and data transmission between the cloud storage system and a storage device according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating a structure of a cloud storage system and data transmission between the cloud storage system and a storage device according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a data storage method of a storage device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application are clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. The following embodiments and their technical features may be combined with each other without conflict.
Fig. 1 is a schematic structural diagram of a memory device according to an embodiment of the invention.
In this embodiment, the storage device is a portable device having a storage space, such as a usb disk, a portable hard disk, a portable device, an SD card, or a TF card. Other non-portable storage devices may also employ the solution of the present invention.
Specifically, the storage device includes: a memory module 110 and a data transmission module 120 connected to the memory module 110.
The storage module 110 is configured to store a local data file and a network address index file, where the network address index file points to a corresponding cloud storage address in a cloud storage space. The storage module 110 includes a nonvolatile storage device, and a flash memory is generally used as a storage medium, so that data interaction with a terminal can be performed, and data storage is easy to carry. The cloud storage address may be a network address, and includes necessary information such as a server ip number, so that the data file can be directly accessed through the cloud storage address.
The data transmission module 120 is configured to perform data transmission between the local storage device and the cloud storage system, and includes: uploading the local data file meeting the preset conditions to a cloud storage system, and forming and editing a network address index file of the cloud storage address.
The data transmission module 120 is further configured to respond to the operation on the network address index file, and access the corresponding cloud storage address and/or download the data file in the corresponding cloud storage address. And uploading or downloading data by means of the network transmission function of the networked data terminal. The operation on the network address index file may be performed by the storage device itself or a terminal device to which the portable device is connected. In one embodiment, the storage device is a mobile phone and has functions of browsing and operating a storage space, and a user can directly access the storage space through the mobile phone to browse, move and the like the address index file. In another embodiment, the storage device may be a usb disk, the usb disk may be connected to a computer, the storage space of the usb disk is accessed through the computer, and the operating system of the computer performs operations such as browsing and moving on the address index file in the usb disk.
The data transmission module 120 may establish a data transmission path between a terminal accessing the internet and a cloud storage system in a wired or wireless manner, or the data transmission module 120 itself has a power supply and a communication function. For example, the data transmission module 120 is connected to a computer already networked through an interface, or has a wireless connection function such as wifi, bluetooth or infrared, and is connected to a terminal device such as a computer or a mobile phone networked through a wireless manner.
The data transmission module 120 may further have a networking function, and may directly access the internet through a WIFI or GSM network to establish a data transmission path with the cloud storage system.
Please refer to fig. 2, which is a schematic diagram illustrating a memory space in the memory module according to an embodiment of the present invention.
In this embodiment, the storage space of the storage module 110 is logically divided into a storage area 111 and a cache area 112.
The storage area 111 is used as a data storage area for storing local data files and network address index files corresponding to the data files stored in the cloud. The data file and the network address index file may be stored in contiguous physical storage spaces, respectively. In some embodiments, the storage area 111 may be logically divided into a local storage area FAT and an INDEX file storage area INDEX for storing local data files and network address INDEX files, respectively, which are convenient for sorting and querying related files.
In this embodiment, the storage module 110 further includes a buffer area 112, where the buffer area 112 may be a physically fixed storage area, or may be a physically continuous or discontinuous storage space of a certain size temporarily allocated by the data transmission module 120 each time data transmission is required. Since the data transmission module 120 is easily limited by bandwidth when transmitting data and the efficiency of real-time data transmission is limited, the data to be transmitted or the downloaded data can be buffered in the buffer area 112 in advance, so as to improve the efficiency of data transmission or downloading. For example, when a video stored in the cloud needs to be played, real-time downloading is easily limited by network speed and bandwidth, and a waiting phenomenon exists; at this time, when the downloaded video file is played locally, the video stream data can be continuously downloaded in the background and cached in the cache region 112 in advance, and when the downloaded video file is played locally subsequently, the data can be directly acquired from the local cache region 112, so that the fluency of video playing is improved. The buffer area 112 may also adopt a storage medium with a data transmission efficiency higher than that of the storage area 111, and may be nonvolatile or volatile, for example, an SRAM may be adopted.
The network address index file stored in the storage module 110 includes at least one of a thumbnail, a two-dimensional code, and a text file, and the network address index file is edited with information such as a hyperlink, points to a cloud storage address of a corresponding data file in a cloud storage system, and can be configured to be clicked, double-clicked, or various operations to access the cloud storage address for file viewing or downloading.
In some embodiments, the network address index file may be a thumbnail, and the thumbnail may have various forms according to different corresponding data files, for example, a thumbnail of a photo, a thumbnail of a certain frame of a video file, a folder shortcut map, and the like, and a user may configure the thumbnail according to specific situations, which is not limited herein. The index file corresponding to the uploaded data file is presented in a thumbnail mode, so that a user can view the storage content of the storage device and think that the file is still stored locally in the sense, and the use experience is improved.
The network address index file has mobility, the network addresses of the data files stored in the cloud terminals have uniqueness, and the network address index file can be moved to other storage devices or terminals to access the storage addresses in the other devices or terminals, so that convenience in data access is provided. Because the network address index file is usually very small, the data file is transferred by transferring the network address index file, so that the efficiency of data migration can be improved.
Fig. 3 is a schematic structural diagram of a memory device according to another embodiment of the invention.
In this embodiment, the storage device further includes a data analysis module 130, which is connected to the storage module 110, and the data analysis module 130 determines whether to upload the data file in the storage module 110 to the cloud storage system according to a preset condition. The data analysis module 130 is connected to the data transmission module 120, and configured to output a corresponding control signal to the data analysis module 130 when it is determined that the data file needs to be uploaded to the cloud storage system. The preset conditions comprise at least one of the file storage time exceeding a first threshold, the file size exceeding a second threshold, the time since the last opening exceeding a third threshold and the device storage capacity being less than a fourth threshold.
In one embodiment, the first threshold may be set to be half a year, 1 year, or the like, and when the storage time of a certain data file in the storage device exceeds the first threshold, the data transmission module 120 uploads the data file when networking is performed next time.
In another embodiment, if a certain data file has exceeded a third threshold, e.g., 3 months, half a year, etc., since the last time the file was opened, indicating that the frequency for using the file is very low, the data transmission module 120 uploads the data file the next time it is networked.
In other embodiments, the data file may be uploaded only when it meets at least two of the conditions, or priorities may be set for the conditions, and whether the data file needs to be uploaded is determined according to the priorities. For example, when the storage capacity of the device is smaller than a fourth threshold as the highest priority, the data file with the file size exceeding the second threshold, or the data file with the storage time exceeding the first threshold, or the data file with the recently opened time exceeding the third threshold is uploaded to the cloud.
The user may set the preset condition according to actual requirements, which is not limited herein.
In some embodiments, the data analysis module 130 is further configured to monitor a size of a storage space of the storage device, and when it is monitored that the remaining storage space of the storage device is insufficient, a data cleaning function is started, and a preset condition is followed in a cleaning process. The difference from the first to fourth thresholds under the preset condition is that in the foregoing embodiment, the storage device stores data statically and uploads data to the cloud, where the size of the storage space is monitored in real time or periodically, the data is stored in the storage device dynamically, and the data is uploaded to the cloud dynamically. The preset conditions here follow the minimum clean and earliest data first clean principles to ensure that as much of the most up-to-date data is retained in the storage device as possible. In some embodiments, the storage device may be used for a vehicle event data recorder, the data analysis module 130 preferentially retains data within a preset time (for example, 1 hour), and after a file is uploaded to the cloud, if the storage space is insufficient, the data that is uploaded at the earliest time is preferentially cleaned according to the sequence of uploading time, the data that has been uploaded before 10 hours is cleaned, and if the space is still insufficient, the data that is not uploaded before 1 hour is cleaned.
Preferably, when the data is cleaned, the data analysis module 130 deletes the uploaded data, but when the data is deleted, necessary data information such as the time, the type, the level, and the like of the cleaned data is recorded in the log system, so that the brief information of the cleaned data file can be subsequently queried, which is also convenient for querying the cleaned data file in the cloud.
Based on the data analysis module 130, the problem that the conventional storage device can only play a role of passive data storage and cannot actively extract and analyze data in the storage device in time is solved, the extraction instantaneity and the analysis efficiency of the data in the storage device are improved, the safety of the data is enhanced, and the data is prevented from being tampered or deleted under the condition of single storage of the data in the storage device.
Furthermore, the storage device in this embodiment can also be the storage device in the internet of things, the internet of vehicles, the smart home, and it can upload data to the cloud through the internet of things channel in the 4G/5G internet of things, the internet of vehicles, the smart home, and the upload process supports breakpoint continuous transmission, can reduce the waste of network traffic and accelerate the upload or download of data.
In some embodiments, the data transmission module 120 of the storage device receives the data file processed by the data analysis module 130, and the data file is uploaded to the cloud. The data transmission module 120 uploads a file through an internet terminal, and preferably, the data analysis module 130 provides a cloud local virtualization mapping function for the terminal, and meanwhile, supports breakpoint continuous transmission, and simplifies the uploading process of the data file.
Further, in some embodiments, any one of JMS, XMPP protocol, AMQP protocol, DDS protocol, MQTT protocol, CoAP protocol, REST/HTTP or C-V2X communication protocol may be used to perform command interaction between the data transmission module 130 and the internet terminal, so as to implement functions of mutual data transmission between both parties and remote data call, and further encrypt and transmit the interactive data on the transmission channel, thereby reducing the security risk of network transmission. And performing a layer of encryption handshake on the communication between the network connection terminal and the data transmission module by adopting an RSA asymmetric encryption algorithm, temporarily negotiating and allocating a two-layer AES symmetric encryption key for each data uploading process, and performing AES encryption transmission on all interactive commands and the uploaded data. Since the mutual key between the internet terminal and the data transmission module 120 is dynamically generated each time, the security of data transmission over the internet can be enhanced.
The uploaded data files can be stored in the cloud storage system in a centralized or distributed mode and stored according to a specific storage framework of the cloud storage system.
Corresponding to the specific storage architecture of the cloud storage system, the data transmission module 120 also has multiple forms when downloading the data file in the cloud storage address, for example, the complete data file stored in a centralized manner can be directly downloaded from the cloud storage system, multiple data blocks stored in a distributed manner can be downloaded from the cloud storage system, and the multiple data blocks are subjected to data splicing to obtain the complete data file.
The data transmission module 120 or the cloud storage system may also perform encryption/decryption operations during uploading or downloading of the data file or the data block, so as to ensure the security of the data.
Data blocks utilizing distributed storage may typically include: under the condition that a certain distributed storage node is not on line, the main data and the redundant data can still be spliced into a complete data file through the data blocks of other storage nodes for downloading or consulting.
In the storage device described in the above embodiment, the storage space of the device is expanded by uploading part of the data file to the cloud, so that the problem of limitation of the storage space of the storage device is solved.
The embodiment of the invention also provides a cloud storage system suitable for data storage of the storage device. The cloud storage system comprises a cloud storage space and a cloud control module, and the cloud storage space is used for storing data files uploaded by the storage equipment; the cloud control module is connected with the cloud storage space and used for establishing data transmission connection with the storage device, distributing a corresponding cloud storage address for the data file uploaded by the storage device and feeding the cloud storage address back to the storage device.
In some embodiments, to increase the speed of data uploading and downloading, the data analysis module 130 may pass each file (data) in the storage device through a string transformation algorithm to generate a unique MD5 value, and then upload the MD5 value through the data transmission module 120. When a new file is required to be uploaded subsequently, comparing the MD5 value of the file to be uploaded with the MD5 value in a cloud database, determining whether the file with the same MD5 value exists, if the file with the same MD5 value does not exist in the cloud database, uploading the file to be uploaded to the cloud database, and if the file with the same MD5 value exists in the cloud database, not uploading the file to be uploaded. The method aims to ensure that the files are not repeatedly uploaded and improve the speed of a user in uploading operation files (repeated files are not uploaded but are displayed to the user to be rapidly and successfully uploaded), save the capacity in the cloud database, improve the data storage utilization rate of the cloud database, ensure the convenience of subsequent database data maintenance due to the uniqueness of each file stored in the cloud database, and do not need to retrieve and delete the repeated files in the database.
Fig. 4 is a schematic structural diagram of a cloud storage system 300 according to an embodiment of the present invention, and a schematic diagram of data transmission between the cloud storage system and the storage device 100.
In this embodiment, the cloud storage system 300 includes a cloud control module 310 and a centralized storage space 320.
The centralized storage space 320 is a central node formed by one or more host computers, data is centrally stored in a memory of the central node, and storage and control processing of the data are completely processed by the cloud control module 310.
The cloud control module 310 receives the data file uploaded from the storage device 100, allocates a storage address in the centralized storage space 320 for the data file, stores the data file, and feeds the cloud storage address back to the storage device, so that the storage device 100 forms a corresponding index file according to the cloud storage address for storage.
The cloud control module 310 is further configured to respond to an access request of an access device to a certain cloud storage address, obtain a corresponding data file from a corresponding address of the storage space 320, and transmit the data file to the access device, where the access device may be the storage device itself or another device that obtains the index file.
The same storage device 100 may establish data transmission paths with the cloud storage system 300 through different terminal devices 200, respectively.
Fig. 5 is a schematic view of a cloud storage system 400 according to another embodiment of the present invention, and a schematic view of data transmission between the cloud storage system and the storage device 100.
The cloud storage system 400 includes a cloud control module 410 and a plurality of distributed storage spaces 420.
The cloud control module 410 is further configured to map the cloud storage address to storage addresses of a plurality of distributed storage spaces, divide the uploaded data file into a plurality of data blocks, and allocate the data blocks to the plurality of distributed storage spaces 420 for distributed storage.
The distributed storage space 420 may be storage spaces in a plurality of computers randomly distributed in space, a processor of a computer may allocate storage addresses for stored data blocks and feed back the storage addresses to the cloud control module 410, and the cloud control module 410 maps the storage addresses of the data blocks in the plurality of distributed storage spaces to a cloud network storage address and feeds back the cloud network storage address to a storage device side to form a corresponding network address index file.
The data block includes main data and redundant data, and the redundant data is used as a data copy, so that when a certain storage space 420 does not access the network or data is lost, missing data can be read from the redundant data of other storage spaces 420.
The access number and the access IP of the distributed storage spaces 420 may change at any time, and storage may be performed only by establishing a connection between the cloud control module 410 and each distributed storage space 420. The cloud control module 410 distributes the data blocks to the distributed storage spaces 420 by shunting. The data blocks stored in each distributed storage space 420 are incomplete, and therefore cannot be read and consulted, and high privacy is achieved.
The distributed storage space 420 may be an idle storage resource of a terminal of another user, and a data block stored in each distributed storage space 420 cannot be read and referred because it is incomplete, so that the distributed storage space has high privacy. And the data block can be encrypted, so that the data security is further enhanced.
When a user accesses a data file through an index file, the cloud control module 410 is further configured to obtain a plurality of corresponding data blocks from the plurality of distributed storage spaces 420, directly download the data blocks to an access device, and splice data by the access device to obtain a complete data file, or, splice the data blocks into a complete data file by the cloud control module 410 and then download the data file to the access device.
The embodiment of the invention also provides a data storage method of the storage device.
Please refer to fig. 6, which is a flowchart illustrating a data storage method of the storage device.
The data storage method comprises the following steps:
and S101, judging whether the data file locally stored in the storage device meets a preset condition or not.
In some embodiments, the preset conditions include: at least one of the file storage time exceeding a first threshold, the file size exceeding a second threshold, the time since last being opened exceeding a third threshold, and the device storage capacity being less than a fourth threshold.
And S102, uploading the data file meeting the preset condition to a cloud storage system, and acquiring a cloud storage address of the data file in a cloud storage space.
In some embodiments, the data files are stored centrally or distributively in a cloud storage system.
In some embodiments, the method of distributed storage comprises: and mapping the cloud storage address into storage addresses of a plurality of distributed storage spaces, dividing the uploaded data file into a plurality of data blocks, and distributing the data blocks to the plurality of distributed storage spaces for distributed storage.
In some embodiments, further comprising: generating a unique MD5 value of each data file by a character string transformation algorithm for the data files in the storage device, comparing the MD5 value of the data file to be uploaded with the MD5 value of each data file in the cloud storage system, and uploading the data file to be uploaded to the cloud storage system when the same MD5 value does not exist in the cloud storage system.
And S103, locally storing a network address index file corresponding to the uploaded data file in the equipment, wherein the network address index file points to the cloud storage address of the data file.
In some embodiments, the network address index file includes at least one of a thumbnail, a two-dimensional code, and a text edited with a hyperlink.
In some embodiments, when the data transmission module performs data transmission, the data to be transmitted is locally cached.
In some embodiments, the data storage method further comprises: and responding to the operation of the network address index file, and accessing the corresponding cloud storage address and/or downloading the file in the corresponding cloud storage address.
In some embodiments, the method for downloading the file in the corresponding cloud storage address comprises: after downloading and acquiring a plurality of distributed storage data blocks, performing data splicing on the plurality of data blocks to acquire a complete data file.
In some embodiments, further comprising: monitoring the size of the storage space of the storage device, and when the monitored residual storage space is smaller than a preset value, cleaning data in the storage space according to preset conditions.
In some embodiments, the data analysis module is further configured to record operation information related to data cleaning in a logging system while cleaning data in the storage space.
The above-mentioned contents related to the embodiments of the storage device and the cloud storage system are all cited herein, and the description of the related features is not repeated.
An embodiment of the present invention further provides a storage device, including: a processor; a memory storing a computer program executable by the processor, the computer program being capable of executing the data storage method described in the above embodiments when executed.
The above-mentioned embodiments are only examples of the present application, and not intended to limit the scope of the present application, and all equivalent structures or equivalent flow transformations made by the contents of the specification and the drawings, such as the combination of technical features between the embodiments and the direct or indirect application to other related technical fields, are also included in the scope of the present application.
Claims (16)
1. A storage device, comprising:
the storage module is used for storing a local data file and a network address index file, and the network address index file points to a corresponding cloud storage address in a cloud storage space;
the data transmission module is used for transmitting data between the local device and the cloud storage system, and comprises: uploading the local data files meeting the preset conditions to a cloud storage system, and forming and editing a network address index file of the cloud storage address.
2. The storage device of claim 1, wherein the data transmission module is further configured to access the corresponding cloud storage address and/or download the data file in the corresponding cloud storage address in response to an operation on the network address index file.
3. The storage device of claim 1, further comprising: and the data analysis module is connected to the storage module and the data transmission module and is used for monitoring the size of the storage space of the storage equipment, and when the monitored residual storage space is smaller than a preset value, the data in the storage space is cleaned according to preset conditions.
4. The storage device of claim 3, wherein the data analysis module is further configured to log data cleaning-related operation information in a logging system while cleaning data in the storage space.
5. The storage device according to claim 3, wherein the data analysis module is further configured to generate a unique MD5 value for each data file by using a string transformation algorithm for the data files in the storage device, compare the MD5 value of the data file to be uploaded with the MD5 value of each data file in the cloud storage system, and control the data transmission module to upload the data file to be uploaded to the cloud storage system when the same MD5 value does not exist in the cloud storage system.
6. The storage device according to claim 1, wherein the data transmission module is configured to establish a data transmission path between an external networked terminal device and the cloud storage system.
7. The storage device according to claim 1, wherein the storage module comprises a buffer area for buffering data when the data transmission module performs data transmission.
8. The storage device according to claim 1, wherein the preset condition includes: at least one of the file storage time exceeding a first threshold, the file size exceeding a second threshold, the time since last being opened exceeding a third threshold, and the device storage capacity being less than a fourth threshold.
9. The storage device according to claim 1, wherein the data file uploaded by the storage device is centrally or distributively stored in the cloud storage system; when downloading the data file in the cloud storage address, the data transmission module is used for directly downloading the complete file from the cloud storage system, or performing data splicing on a plurality of data blocks after acquiring the plurality of data blocks to acquire the complete data file.
10. A data storage method of a storage device, comprising:
judging whether a data file locally stored by the equipment meets a preset condition or not;
uploading the data file meeting the preset condition to a cloud storage system, and acquiring a cloud storage address of the data file in a cloud storage space;
and locally storing a network address index file corresponding to the uploaded data file in the equipment, wherein the network address index file points to the cloud storage address of the data file.
11. The data storage method of claim 10, further comprising: and responding to the operation of the network address index file, and accessing the corresponding cloud storage address and/or downloading the data file in the corresponding cloud storage address.
12. The data storage method of claim 10, further comprising: generating a unique MD5 value of each data file by a character string transformation algorithm for the data files in the storage device, comparing the MD5 value of the data file to be uploaded with the MD5 value of each data file in the cloud storage system, and uploading the data file to be uploaded to the cloud storage system when the same MD5 value does not exist in the cloud storage system.
13. The data storage method of claim 10, further comprising: monitoring the size of the storage space of the storage device, and when the monitored residual storage space is smaller than a preset value, cleaning data in the storage space according to preset conditions.
14. The data storage method of claim 13, wherein the data analysis module is further configured to perform a cleaning operation on the data in the storage space while recording operation information related to the cleaning operation on a logging system.
15. A storage device, comprising:
a processor;
memory storing a computer program executable by the processor, the computer program being capable of performing the data storage method of any one of claims 10 to 14 when executed.
16. A cloud storage system, comprising:
the cloud storage space is used for storing the data file uploaded by the storage device of any one of claims 1 to 9;
and the cloud control module is connected with the cloud storage space and used for establishing data transmission connection with the storage equipment, distributing a corresponding cloud storage address for the data file uploaded by the storage equipment, and feeding the cloud storage address back to the storage equipment end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110298190.XA CN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110298190.XA CN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114610679A true CN114610679A (en) | 2022-06-10 |
Family
ID=81857380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110298190.XA Pending CN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114610679A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595007A (en) * | 2023-05-23 | 2023-08-15 | 建材广州工程勘测院有限公司 | Geotechnical engineering geological data management system |
-
2021
- 2021-03-19 CN CN202110298190.XA patent/CN114610679A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595007A (en) * | 2023-05-23 | 2023-08-15 | 建材广州工程勘测院有限公司 | Geotechnical engineering geological data management system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110401724B (en) | File management method, file transfer protocol server and storage medium | |
| US20080189294A1 (en) | Method and apparatus for sharing content | |
| CN101277211A (en) | Method and apparatus for buffering data | |
| WO2020098654A1 (en) | Data storage method and device based on cloud storage, and storage medium | |
| WO2017206754A1 (en) | Storage method and storage device for distributed file system | |
| US20170153909A1 (en) | Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine | |
| US20030154246A1 (en) | Server for storing files | |
| US10284645B1 (en) | Backup from network attached storage to sequential access media in network data management protocol environments | |
| US20090327303A1 (en) | Intelligent allocation of file server resources | |
| CN114610679A (en) | Storage device, data storage method thereof and cloud storage system | |
| JP2009157761A (en) | Storage system and data management method thereof | |
| US20140115091A1 (en) | Machine-implemented file sharing method for network storage system | |
| CN103973772B (en) | File read/write method and device | |
| US9432238B2 (en) | Communicating large amounts of data over a network with improved efficiency | |
| WO2023093608A1 (en) | Automatic distributed cloud storage scheduling interaction method and apparatus, and device | |
| CN110737635B (en) | Data blocking method | |
| US20220075886A1 (en) | File interface to dynamically manage access to remote files | |
| JP2004139200A (en) | File management program and file management system | |
| US10735512B2 (en) | Managing data | |
| JP2019532399A (en) | Data replication in scalable messaging systems | |
| KR100974916B1 (en) | Virtual disk drive file transfer system and the method thereof | |
| JPH11345179A (en) | Method and system for managing data, constituent apparatus and recording media | |
| Dai et al. | Dasc: A privacy-protected data access system with cache mechanism for smartphones | |
| US20160173381A1 (en) | Nas off-loading of network traffic for shared files | |
| US20220408126A1 (en) | System and method for optimizing the distribution of available media production resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |