WO2019057193A1 - 数据删除方法及分布式存储*** - Google Patents

数据删除方法及分布式存储*** Download PDF

Info

Publication number
WO2019057193A1
WO2019057193A1 PCT/CN2018/107277 CN2018107277W WO2019057193A1 WO 2019057193 A1 WO2019057193 A1 WO 2019057193A1 CN 2018107277 W CN2018107277 W CN 2018107277W WO 2019057193 A1 WO2019057193 A1 WO 2019057193A1
Authority
WO
WIPO (PCT)
Prior art keywords
partition
data
storage
deleted
time
Prior art date
Application number
PCT/CN2018/107277
Other languages
English (en)
French (fr)
Inventor
黄华东
夏伟强
王伟
林起芊
Original Assignee
杭州海康威视***技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视***技术有限公司 filed Critical 杭州海康威视***技术有限公司
Publication of WO2019057193A1 publication Critical patent/WO2019057193A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608Saving storage space on storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Definitions

  • the present application relates to the field of data processing technologies, and in particular, to a data deletion method and a distributed storage system.
  • a distributed storage system is a system that interconnects and works with different servers through a network to provide users with massive data storage functions.
  • the distributed storage system mainly includes: a management server and a storage server, and the management server uses the index information of the stored data, and the index information of the data includes information such as the size of the data, the storage time, and the storage location.
  • the index information of the data further includes index information of the data block corresponding to the data;
  • the management server may be a metadata server (Metadata Server (MDS) in the object storage system; the storage server is mainly used to store data, exemplary
  • the storage server can be an Object-based Storage Device (OSD).
  • OSD Object-based Storage Device
  • the data deletion method in the related art is: the management server periodically determines the expired data according to the preset storage period and the storage time of the data; and determines, for each data in the determined expired data, the storage server storing the data. And generating a delete instruction for each data block of the data; sending a delete instruction to the determined storage server to cause the storage server receiving the delete instruction to delete the data block corresponding to the delete instruction.
  • the management server may delete the index information of the determined data after sending a delete instruction to the storage server.
  • the management server generates a delete instruction for a data block, and sends the delete instruction to the storage server through the network protocol, and transmits the instruction through the network protocol, and the network interaction takes a certain time, when there is a large number of deletions In the case that an instruction needs to be sent, a large amount of network interaction is generated. These network interactions consume a lot of time, which results in slower deletion of data in batches and inefficient storage of storage space.
  • the purpose of the embodiment of the present application is to provide a data deletion method and a distributed storage system, so as to improve the efficiency of releasing a storage space.
  • the specific technical solutions are as follows:
  • the embodiment of the present application provides a data deletion method, which is applied to a management server in a distributed storage system, and the method includes:
  • the target time point is determined according to a preset data deletion rule
  • the method further includes:
  • the standard time is obtained according to the system time of each storage server, where the standard time is a time after the system time of each storage server is synchronized;
  • the storage time is a standard time when the storage server stores data
  • the target time point is the latest storage time of the storage time of the data to be deleted.
  • the timing is based on a system time of each storage server, and the standard time is obtained, including:
  • the determining, according to the preset data deletion rule, the target time point including:
  • the method further includes:
  • the target time point and the corresponding partition identifier delete the data to be deleted stored in the corresponding first partition.
  • the determining, according to the preset data deletion rule of each partition, a target time point of each first partition to be deleted including:
  • the difference between the obtained current time and the preset storage period of the first partition is taken as the target time point of the first partition.
  • the determining, according to the preset data deletion rule of each partition, a target time point of each first partition to be deleted including:
  • the latest storage time is selected from the storage time of all data to be deleted determined for the first partition as the target time point of the first partition.
  • the method further includes:
  • the index information of the data stored before the target time point is deleted.
  • the deleting the index information of the data stored at the target time point includes:
  • index information for data stored by the first partition before the target time point of the first partition is deleted.
  • the embodiment of the present application further provides a data deletion method, which is applied to a storage server in a distributed storage system, where the method includes:
  • the embodiment of the present application further provides a data deletion method, which is applied to a distributed storage system, where the distributed storage system includes a management server and a storage server, and the method includes:
  • the target time point is determined according to the preset data deletion rule; and the determined target time point is sent to the storage server;
  • the data to be deleted is determined according to the target time point; and the determined data to be deleted is deleted, wherein the data to be deleted is the storage time before the target time point.
  • the stored data is the
  • the embodiment of the present application discloses a distributed storage system, where the system includes a management server and a storage server.
  • the management server is configured to: when determining that the data needs to be deleted in batches, determine a target time point according to a preset data deletion rule; and send the determined target time point to the storage server;
  • the storage server is configured to: after the target time point is received, determine data to be deleted according to the target time point; delete the determined data to be deleted, where the data to be deleted is stored in the Stored data before the target time point.
  • an embodiment of the present application further provides a management server, including: a first processor, a first communication interface, a first memory, and a first communication bus, wherein the first processor The first communication interface and the first memory complete communication with each other through the first communication bus;
  • the first memory is configured to store a computer program
  • the first processor is configured to execute a computer program stored on the first memory to implement any of the data deletion method steps applied to the management server.
  • an embodiment of the present application further provides a storage server, including: a second processor, a second communication interface, a second memory, and a second communication bus, wherein the second processor The second communication interface and the second memory complete communication with each other through the second communication bus;
  • the second memory is configured to store a computer program
  • the second processor is configured to execute a computer program stored on the second memory to implement any of the data deletion method steps applied to the storage server.
  • an embodiment of the present application further provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is implemented by a processor to implement the foregoing application. Any data deletion method step on the management server.
  • an embodiment of the present application further provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is implemented by a processor to implement the foregoing application. Any data deletion method step for the storage server.
  • an embodiment of the present application further provides an application, where the application is configured to execute any of the data deletion method steps applied to the management server at runtime.
  • an embodiment of the present application further provides an application, where the application is configured to execute any of the foregoing data deletion method steps applied to the storage server at runtime.
  • the management server may send a target time point to the storage server, so that the storage server may delete the data to be stored according to the target time point, and does not need to send a delete instruction for each data block.
  • Reduces a large amount of network interaction thus reducing the time spent on network interaction between the management server and the storage server, and improving the efficiency of storage space release.
  • implementing any of the products or methods of the present application necessarily does not necessarily require all of the advantages described above to be achieved at the same time.
  • FIG. 1 is a schematic flowchart of a data deletion method according to an embodiment of the present application
  • FIG. 2 is still another schematic flowchart of a data deletion method according to an embodiment of the present application.
  • FIG. 3 is another schematic flowchart of a data deletion method according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of another data deletion method according to an embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of a distributed storage system provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a data deletion apparatus according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of another data deletion apparatus according to an embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a management server according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a storage server according to an embodiment of the present application.
  • the present disclosure provides a data deletion method and a distributed storage system, in order to solve the technical problem that the efficiency of the storage of the data in the related art is relatively slow and the storage space is not efficient.
  • a data deletion method provided by the embodiment of the present application is described in detail below.
  • the data deletion method provided by the embodiment of the present application may be preferably applied to a management server in a distributed storage system, or may be preferably applied to a storage server in a distributed storage system, and may also be preferably applied to a distributed storage system.
  • the distributed storage system includes a management server and a storage server.
  • the management server is mainly used for a server that stores index information of data, and the storage server is mainly used for storing data.
  • the target time point is determined according to the preset data deletion rule, and the determined target time point is sent to the storage server, and the storage server follows the target time point. Delete the data to be deleted.
  • the management server does not need to send delete instructions for each data block, which reduces a large amount of network interaction, thereby reducing the time consumed by the network interaction between the management server and the storage server, and improving the efficiency of the storage space release.
  • the distributed storage system may be an object storage system, and the object storage system may provide users with massive, secure, highly reliable, and scalable cloud storage services.
  • management The server can be an MDS and the storage server can be an OSD.
  • FIG. 1 is a schematic flowchart of a data deletion method according to an embodiment of the present disclosure, which is applied to a management server in a distributed storage system, and the method includes:
  • the target time point is determined according to a preset data deletion rule.
  • the first type According to the preset time interval, when the distance is determined last time whether the data needs to be deleted in batches reaches a preset time interval, it is determined whether the data needs to be deleted in batches.
  • the preset time interval may be determined based on the amount of data to be stored in the preset duration and the storage capacity of the storage server, or may be determined according to the type of the stored data, and may be determined according to other manners. Not limited. Exemplarily, if the stored data is a monitoring video, a large amount of video data is generated within a preset duration, and the time interval may be set shorter; if the stored user's shopping data, because each shopping data occupies storage space They are all small, and the time interval can be set longer; in the case of stored video data, shopping data, and other types of data, the time interval can be set according to actual conditions.
  • the management server receives the command sent by the user.
  • the instruction mentioned here is a batch delete instruction. After receiving the instruction, the management server determines the target time point according to the preset data deletion rule.
  • the data deletion rule may be set in advance according to actual conditions. Specifically, it may be determined based on a preset storage period, or may be determined based on a preset storage threshold.
  • data can also be partitioned for storage.
  • the target time point of the partition is required, and the target time point is determined according to a preset data deletion rule, which may include:
  • the target time point of the first partition of each data to be deleted is determined according to a preset data deletion rule of each partition.
  • the partition mentioned here is a logical partition, which can be a partition for the user, one partition corresponds to one user, and one user can correspond to multiple partitions. Data belonging to the same partition can be stored in different storage servers. Different data deletion rules can be set for different partitions, and the same data deletion rules can be set.
  • the first partition refers to a partition whose amount of data to be deleted is not zero. If the amount of data to be deleted of a partition is zero, it indicates that there is no data to be deleted in the partition, further indicating that the target time cannot be determined according to the data deletion of the partition. point.
  • the partition identifier may be the name of the partition or the number of the partition, and the specific partition identifier may include at least one of a character, a number, a letter, and a character.
  • the partition ID is pre-stored in the database or it can be stored in the memory of the management server.
  • S102 Send the determined target time point to the storage server, so that the storage server that receives the target time point deletes the data to be deleted according to the target time point, wherein the data to be deleted is the storage time in the Stored data before the target time point.
  • the data to be deleted may be determined and the index information to be deleted may be deleted before being sent to the storage server after determining the target time point.
  • the storage server After receiving the target time point, the storage server determines the data to be deleted according to the target time point, and deletes all the determined data to be deleted.
  • the method for determining the data to be deleted by the storage server may be: determining whether the storage time of the data recorded by itself is stored no later than the target time point, and if so, the data corresponding to the storage time of the target time point , determined as the data to be deleted.
  • the target time point of each first partition and the corresponding partition identifier may be sent to the storage server corresponding to each first partition, so that the target of the first partition is received.
  • the storage server at the time point and the corresponding partition identifier deletes the data to be deleted stored in the corresponding first partition according to the target time point of the first partition and the corresponding partition identifier.
  • the storage server After receiving the target time point of the first partition and the corresponding partition identifier, the storage server determines the first partition corresponding to the partition identifier, and determines, according to the pre-recorded storage time of the stored data of the first partition.
  • the data stored before the target time point of the first partition is used as the data to be deleted corresponding to the first partition, and the determined data to be deleted corresponding to the first partition is deleted.
  • the corresponding partition identifier referred to herein refers to the partition identifier of the first partition corresponding to the target time point of the first partition that is sent.
  • the target time point of the first partition and the partition identifier of the first partition may be separately sent to the storage server, or the target time point of the first partition and the partition of the first partition may be The identity is sent to the storage server as a whole.
  • the target time point of one first partition and the partition identifier of the first partition may be sent to the storage server as a whole.
  • the storage server receives a delete instruction, deletes the stored data block corresponding to the delete instruction, and generates a large amount of network interaction between the storage server and the management server, and the network interaction consumes a large amount of time, resulting in data deletion.
  • the speed is slower. If the data is stored very fast, the data storage speed is likely to be greater than the data deletion speed. In this case, the storage space of the storage server is continuously occupied, which may result in insufficient storage space and insufficient storage space, which may cause subsequent storage of data to be stored to fail.
  • the management server sends the target time point to the storage server, and the storage server deletes the data stored before the target time point to reduce the time consumed by the network interaction between the management server and the storage server, and reduces the storage server.
  • the time to wait for the instruction increases the speed of data deletion. Compared with the related art, at the same storage speed, there is no shortage of space, and all the data to be stored can be stored as much as possible.
  • the management server after determining the target time point, sends the determined target time point to the storage server, so that the storage server can delete the data to be deleted according to the target time point, specifically, is stored before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes one data block, which can reduce the network interaction between the management server and the storage server and consumes The time is reduced, the time for the storage server to wait for the instruction is increased, the data deletion speed is increased, and the efficiency of the storage space release is improved, thereby improving the efficiency of the storage space release.
  • the target time point may be determined according to a standard time synchronized with the storage server system time. For example, the following steps can be taken to determine the standard time:
  • the standard time is obtained according to the system time of each storage server, where the standard time is a time after the system time of each storage server is synchronized;
  • the storage time is a standard time when the storage server stores data
  • the target time point is the latest storage time of the storage time of the data to be deleted.
  • the duration of the timing may be the same as or different from the preset time interval, and is not limited herein.
  • the duration of the specific timing may be determined according to actual conditions.
  • the system time of the storage server refers to the current system time of the storage server. It can be sent by the storage server to the management server, or the system time of other servers collecting the storage server, and the collected system time is sent to the management server.
  • the storage server reports its own system time to the management server.
  • the management server immediately saves the obtained system time to the memory, and synchronizes the system time of each storage server to obtain the standard time.
  • the standard time is the storage server system time.
  • the time after the synchronization process the difference between the standard time and the system time of the storage server is within the preset time threshold. In general, the difference between the standard time and the system time of most storage servers is Second level.
  • the system time of each storage server may be synchronized to obtain the standard time by using the following steps:
  • the system time of the storage server may be deviated.
  • the specific method can be: When it is determined that the number of all acquired system time is greater than the preset number, the maximum system time and a minimum system time are eliminated, and the remaining system time is sorted according to the size, and the system time is ranked in the middle.
  • As standard time this can avoid the system time of the deviation and affect the accuracy of the target time point obtained.
  • the system time of most storage servers differs within a preset time range.
  • the average of the remaining system time can also be calculated as the quasi-time, which can also avoid the system time of the deviation affecting the accuracy of the target time point obtained.
  • the above maximum system time is the earliest system time; the minimum system time is the latest system time.
  • the above-mentioned system time in the middle is the system time in which the ranking is in the middle.
  • the above-mentioned system time in the middle may be one of the two system times ranked in the middle, or The average of the two system times ranked in the middle.
  • the average of all the obtained system times can be directly taken as the standard time.
  • the storage server may be faulty, causing the storage server to be offline, the system time of the online storage server is obtained, and the obtained system time may be changed, and the storage server is obtained.
  • the standard time obtained each time can be saved, and the saved standard time is obtained when the management server is restarted.
  • the standard time obtained is saved. Specifically, the obtained standard time can be saved to the database.
  • the data base refers to an associated structured data set that is reasonably stored on the storage device of the computer, and a database. Contains various parts, including tables, views, stored procedures, records, fields, indexes, and more.
  • the storage process may include a storage manner, a data identifier, and the like.
  • the standard time for saving the obtained storage server can also be easily read when the management server is restarted.
  • the system time of the storage server may not be obtained immediately, thereby obtaining the standard time, but it is possible to receive the data to be stored.
  • the standard time of the stored storage server can be obtained, and the currently stored standard time is used as the storage time of the data to be stored, and the storage time of the data recorded by the management server is consistent with the storage time of the data recorded by the storage server. .
  • the main purpose of obtaining the standard time is to make the storage time of the data recorded in the index information of the data held by the management server coincide with the storage time of the data recorded by the storage server.
  • the time step is added to the currently saved standard time according to the preset time step, as the current standard time, the management server
  • the index information of the data is saved, the storage time of the data is described as the current standard time.
  • the system time synchronization of the storage server is required, and the storage server system time synchronization is:
  • the system time of each storage server can be synchronized with the system time of the NTP server through the network time protocol NTP.
  • NTP Network Time Protocol
  • the storage server establishes a communication connection with the NTP server, and synchronizes the system time of the system with the system time of the NTP server.
  • the NTP server mentioned here can be a professional NTP server.
  • it can be an NTP server of the National Time Service Center server; it can also be a server specially set up in a distributed storage system.
  • Server A is the storage server. Before the system time of the NTP server is synchronized, the system time of server A is 10:00:00 AM, and the system time of the NTP server is 11:00:00 AM.
  • the specific working process of time synchronization is:
  • Server A sends an NTP message packet to the NTP server.
  • the NTP message packet carries the timestamp when the NTP message packet leaves the server A.
  • the timestamp is 10:00:00 (t1).
  • the NTP server adds its own timestamp, which is 11:00:01 (t2).
  • the NTP server adds its own timestamp, which is 11:00:02 (t3).
  • server A When server A receives the response message packet, it adds a new timestamp, which is 10:00:03 (t4).
  • the standard time after synchronizing the system time of each storage server is obtained, which can ensure that the storage time used by the management server to record the index is synchronized with the storage time recorded when the storage server stores the data, thereby further ensuring Delete the accuracy of the data.
  • the target time point may be determined according to the system time of the management server.
  • the storage time synchronization can synchronize the management server and the storage server with the system time of the NTP server through the network time protocol NTP, and the system time of the management server and the system time of the NTP server can be synchronized with the system time of the storage server and the NTP server.
  • the system time synchronization is the same, and will not be described here.
  • the management server and the storage server synchronize the system time with the NTP server, and can further ensure that the storage time used by the management server to record the index information is synchronized with the storage time recorded when the storage server stores the data, thereby further Guarantee the accuracy of deleting data.
  • the management server stores the index information of each data. After the storage server deletes the data, in order to save the storage space of the management server, and in order to reduce the waiting time of the user when the user requests to read the deleted data, the target can be deleted. Index information of data stored before the time point. Because the index information includes the last modification time of the data, that is, the storage time of the data, the index information corresponding to the data before the target time point of the data storage time may be determined as the index information to be deleted, and then the index information to be deleted is deleted.
  • deleting index information of data stored before a target time point includes:
  • index information for data stored by the first partition before the target time point of the first partition is deleted.
  • the first partition is a partition whose amount of data to be deleted is not zero. It indicates that the data to be deleted that has been stored in the first partition needs to be deleted, and the index information of the data to be deleted that has been stored in the first partition needs to be deleted.
  • the management server determines, from the index information of the data stored for the first partition, index information of data stored before the target time point of the first partition as the index information of the first partition to be deleted, and deletes the first The index information of the partition to be deleted.
  • the time used for the information is the same as the time taken by the storage server to store the data.
  • the time used for recording the index information may be the currently stored standard time
  • the time used by the storage server to record the storage time of the data may be the system time of the storage server, the system time of the storage server, and the NTP server.
  • the system time is kept in sync; in addition, when the system time of the management server and the system time of the storage server are synchronized with the system time of the NTP server, the time used to record the index information may be the system time of the management server, and the storage server records The time taken for the storage time of the data can be the system time of the storage server.
  • the management server deletes the index information stored before the target time point, which not only saves the storage space of the management server.
  • the management server and the storage server perform the delete operation concurrently, which can avoid the request waiting, greatly improve the efficiency of the storage space release, and avoid the problem that the latest data to be stored is lost due to the slow release of space caused by the unreleased disk space.
  • a data deletion method provided by an embodiment of the present application is described below in conjunction with another specific embodiment.
  • FIG. 2 is still another schematic flowchart of a data deletion method according to an embodiment of the present disclosure, where the method includes:
  • the current time can be the system time of the management server or the standard time currently stored in the database.
  • the standard time currently stored in the database may be a standard time obtained based on the system time of the storage server, or may be an increase in the preset time step based on the standard time obtained based on the system time of the storage server.
  • the time step is obtained.
  • the system time of the management server mentioned here may be the system time after synchronization with the system time of the storage server.
  • the storage period of the partition is set according to the user's needs. Different partitions can be set to the same storage period or different storage periods.
  • the target time point of the first partition is obtained.
  • the current time is 9:00 am on August 15, 2017, the first partition is partition A, and the preset storage period of partition A is 10 days, then the calculated target time of partition A is 2017. At 9:00 am on August 5th.
  • the difference between the current time and the storage period of the first partition as the target time point of the first partition may be understood as determining the target time point by using the period coverage logic principle.
  • S203 Acquire a partition identifier of each first partition.
  • S204 Send the target time point of each first partition and the corresponding partition identifier to the storage server corresponding to each first partition, so that the storage server that receives the target time point of the first partition and the corresponding partition identifier is The data to be deleted stored in the corresponding first partition is deleted according to the target time point of the first partition and the corresponding partition identifier.
  • the management server determines the target time point of the first partition by using the periodic coverage principle, and sends the determined first partition target time point and the corresponding partition identifier to the storage server, so that the storage server can delete the first partition.
  • the stored data to be deleted can delete multiple data blocks at a time.
  • the storage server receives a deletion instruction and deletes one data block.
  • the management server and the storage server can be reduced. The time spent between network interactions reduces the time that the storage server waits for instructions, increases the speed of data deletion, and improves the efficiency of storage space release.
  • a data deletion method provided by an embodiment of the present application is described below in conjunction with another specific embodiment.
  • FIG. 3 is another schematic flowchart of a data deletion method according to an embodiment of the present disclosure, where the method includes:
  • the management server stores the index information of the data stored in each partition.
  • the index information includes the data size of each data.
  • the current storage capacity of the partition can be calculated.
  • the management server can also record The current storage capacity of the partition.
  • the management server increases the size of the data in the recorded storage capacity of the partition.
  • the storage threshold of each partition is set according to the actual situation in advance, and the storage threshold may be a fixed value, or may be determined according to the total storage capacity of the partition and the storage storage threshold preset by the partition, for example, in advance
  • the storage threshold of the partition B is set to 15G, and the current storage capacity of the partition B is 20G, so that the storage capacity of the partition B to be deleted is 5G; the total storage capacity of the partition C is 15G, and the preset storage ratio threshold is 80%.
  • the storage threshold of the partition C is 12G, and the storage capacity of the partition C to be deleted is determined to be 3G.
  • S302 for each first partition where the storage capacity to be deleted is not zero, according to the pre-recorded data size of the stored data of the first partition, according to the pre-recorded storage time of the stored data of the first partition, Determining the data to be deleted of the first partition until the determined difference between the total amount of data of all the data to be deleted and the amount of the to-be-deleted storage of the first partition is within a preset data range.
  • the partition cannot be called the first partition.
  • the management server determines the data to be deleted in the first partition according to the order of the storage time of the stored data in the first partition, and the specific implementation manner may be: determining the data with the earliest storage time as the data to be deleted. Data, determining whether the total amount of the quantity to be deleted and the amount of the to-be-deleted storage of the first partition are within a preset data range, and the preset data range may be the same for different partitions, or may be different for the same For the partition, the preset data range may also be different; if not, the other data stored in the storage other than the data to be deleted is determined as the data to be deleted, and the number of all the to-be-deleted is determined. Whether the total amount is different from the amount of the to-be-deleted storage of the first partition is within a preset data range; if yes, executing S303.
  • the determined total amount of data of the data to be deleted has a high probability that it is not equal to the amount of data to be deleted of the first partition. Therefore, the following conditions are required to continue to determine the data to be deleted:
  • the difference between the determined total amount of data of the data to be deleted and the amount of the to-be-deleted storage of the first partition is within a preset data range.
  • S303 For each first partition, select the latest storage time from the storage time of all the data to be deleted determined for the first partition as the target time point of the first partition.
  • the data to be deleted determined for each first partition is determined according to the storage chronological order, and the latest storage time is selected as the target time point of the first partition, which can be understood as the storage time of the last determined data to be stored. As the target time point of the first partition.
  • the storage time of all data to be deleted is sorted chronologically, and the latest storage time is selected as the target time point of the first partition.
  • the target time point for determining the first partition is determined by the capacity coverage logic.
  • the capacity coverage is based on the set capacity threshold, and the earliest stored data is deleted to achieve the capacity meets the setting requirements.
  • S304 Acquire a partition identifier of each first partition.
  • S305 Send the target time point of each first partition and the corresponding partition identifier to the storage server corresponding to each first partition, so that the storage server that receives the target time point of the first partition and the corresponding partition identifier, according to the The target time point of a partition and the corresponding partition identifier delete the data to be deleted stored in the corresponding first partition.
  • the management server determines the target time point of the first partition by using the capacity coverage principle, and sends the determined first partition target time point and the corresponding partition identifier to the storage server, so that the storage server can delete the first partition.
  • the stored data to be deleted can delete multiple data blocks at a time.
  • the storage server receives a deletion instruction and deletes one data block.
  • the management server and the storage server can be reduced. The time spent between network interactions reduces the time that the storage server waits for instructions, increases the speed of data deletion, and improves the efficiency of storage space release.
  • FIG. 4 is a schematic flowchart of another data deletion method according to an embodiment of the present disclosure, which is applied to a storage server in a distributed storage system, and the method includes:
  • S401 Receive a target time point sent by the management server, where the target time point is determined according to a preset data deletion rule when the management server determines that the data needs to be deleted in batches.
  • the target time point sent by the receiving management server may include:
  • the storage server receives the partition identifier of the first partition while receiving the target time point of the first partition.
  • S402 Determine data to be deleted according to the target time point.
  • the storage server may determine whether there is data stored before the target time point time according to the storage time of each data recorded by itself, and if so, determine the data stored before the target time point to be determined. delete data.
  • determining the data to be deleted according to the target time point may include:
  • the data that the first partition has stored before the target time point of the first partition is determined as the data to be deleted of the first partition.
  • the data of a first partition can be stored in one storage server or in multiple storage servers.
  • a storage server can store data of one partition or store data of multiple partitions.
  • the storage server After receiving the target time point of the first partition and the corresponding partition identifier, the storage server first determines, according to the partition identifier of the first partition, whether the data corresponding to the first partition is stored, and if so, the first partition is in the The data stored before the target time point of the first partition is determined as the data to be deleted of the first partition.
  • the first partition is partition 1
  • the partition 1 partition identifier is A1
  • the storage server records the partition identifier including A1
  • the storage server stores the data corresponding to the partition 1 and the partition 1 has stored data.
  • the data stored before the target time point of the partition 1 is determined as the data to be deleted of the partition 1.
  • deleting the determined data to be deleted includes:
  • the determined data to be deleted for each of the first partitions is deleted.
  • the storage server may determine the data to be deleted according to the target time point, and delete the determined data to be deleted, specifically, the data stored before the target time point is deleted, that is, multiple data blocks may be deleted.
  • the storage server receives a delete instruction and deletes a data block, the time consumed by the network interaction between the management server and the storage server can be reduced, and the time for the storage server to wait for the instruction is reduced, and the time is increased.
  • the speed of data deletion which in turn increases the efficiency of storage space release.
  • the method may further include:
  • the system time of the storage server itself is periodically sent to the management server, so that the management server periodically obtains the standard time according to the system time of each storage server, where the standard time is after the system time of each storage server is synchronized. time.
  • the system time of the storage server is synchronized with the system time of the NTP server through the NTP.
  • the system time of each storage server can be synchronized by the system time of the NTP server, and the synchronization method is the same as the synchronization method described above, and is not described herein.
  • FIG. 5 is a schematic flowchart of another data deletion method according to an embodiment of the present disclosure.
  • the data deletion method is applied to a distributed storage system, where the distributed storage system includes a management server and a storage server, and the method includes:
  • the management server determines a target time point according to a preset data deletion rule when determining that the data needs to be deleted in batches, and sends the determined target time point to the storage server.
  • the management server determines the target time point according to the preset data deletion rule, including:
  • the management server sends the determined target time to the storage server, including:
  • the management server determines, according to a preset data deletion rule of each partition, a target time point of each first partition to which data is to be deleted, including:
  • the difference between the obtained current time and the preset storage period of the first partition is taken as the target time point of the first partition.
  • the management server determines, according to a preset data deletion rule of each partition, a target time point of each first partition to which data is to be deleted, including:
  • the latest storage time is selected from the storage time of all data to be deleted determined for the first partition as the target time point of the first partition.
  • the management server deletes index information of data stored before the target time point.
  • the management server deletes the index information of the data stored before the target time point, including:
  • index information for data stored by the first partition before the target time point of the first partition is deleted.
  • the storage server determines the data to be deleted according to the target time point; and deletes the determined data to be deleted, where the data to be deleted is the storage time before the target time point. The data has been stored.
  • the storage server receives a target time point of the first partition sent by the management server and a corresponding partition identifier
  • the storage server determines the data to be deleted according to the target time point, including:
  • the storage server deletes the determined data to be deleted, including:
  • the determined data to be deleted for each of the first partitions is deleted.
  • the storage server periodically sends the system time of the storage server itself to the management server;
  • the management server periodically obtains a standard time according to a system time of each storage server, where the standard time is a time after synchronizing the system time of each storage server;
  • the storage time is a standard time when the storage server stores data
  • the target time point is the latest storage time of the storage time of the data to be deleted.
  • the management server periodically obtains standard time according to the system time of each storage server, including:
  • the management server after determining the target time point, sends the determined target time point to the storage server, and the storage server may delete the data to be deleted according to the target time point, specifically, the data to be stored before the target time point. All deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes one data block, which can reduce the network interaction between the management server and the storage server. Time, reducing the time that the storage server waits for instructions, increasing the speed of data deletion, thereby increasing the efficiency of storage space release.
  • FIG. 6 is a schematic structural diagram of a distributed storage system according to an embodiment of the present disclosure.
  • the distributed storage system includes a management server 610 and a storage server 620, where
  • the management server 610 is configured to: when it is determined that the data needs to be deleted in batches, determine a target time point according to a preset data deletion rule; and send the determined target time point to the storage server 620;
  • a storage server 620 configured to: after the target time point is received, determine data to be deleted according to the target time point; delete the determined data to be deleted, where the data to be deleted is a storage time at the target Stored data before the point in time.
  • the management server after determining the target time point, sends the determined target time point to the storage server, and the storage server may delete the data to be deleted according to the target time point, specifically, stored before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes one data block, which can reduce the network interaction between the management server and the storage server and consumes Time, reducing the time that the storage server waits for instructions, increasing the speed of data deletion, thereby increasing the efficiency of storage space release.
  • the storage server 620 is configured to periodically send the system time of the storage server 620 to the management server 610;
  • the management server 610 is configured to obtain a standard time according to a system time of each storage server 620, where the standard time is a time after synchronizing the system time of each storage server 620;
  • the storage time is a standard time when the storage server 620 stores data
  • the target time point is the latest storage time of the storage time of the data to be deleted.
  • the management server 610 periodically obtains the standard time according to the system time of each storage server 620, including:
  • the management server 610 is configured to determine, according to a preset data deletion rule of each partition, a target time point of each first partition to which data is to be deleted; and acquire each of the first partitions. a partition identifier; the target time point of each first partition and the corresponding partition identifier are sent to the storage server 620 corresponding to each first partition;
  • the storage server 620 is configured to receive a target time point of the first partition and the corresponding partition identifier sent by the management server 610. For each first partition, determine the storage server 620 according to the partition identifier of the first partition. Whether the data corresponding to the first partition is stored; if yes, determining the data that the first partition has stored before the target time point of the first partition as the data to be deleted of the first partition; deleting the determined data Data to be deleted for each first partition.
  • the management server 610 is configured to determine, according to a preset data deletion rule of each partition, a target time point of each first partition to be deleted, including:
  • the difference between the obtained current time and the preset storage period of the first partition is taken as the target time point of the first partition.
  • the management server 610 is configured to determine, according to a current storage quantity of each partition and a storage threshold preset for each partition, a storage quantity to be deleted for each partition; For each first partition whose quantity is not zero, according to the pre-recorded data size of the stored data of the first partition, the first partition is determined according to the pre-recorded storage time of the stored data of the first partition. Data to be deleted until the determined difference between the total amount of data of all data to be deleted and the amount of storage to be deleted of the first partition is within a preset data range; for each first partition, determined for the first partition In the storage time of all the data to be deleted, the latest storage time is selected as the target time point of the first partition.
  • the management server 610 is configured to delete index information of data stored before the target time point.
  • the management server 610 is configured to, for each first partition, delete index information of data stored before the target time point of the first partition for the first partition.
  • the embodiment of the present application further provides a data deletion setting.
  • FIG. 7 is a schematic structural diagram of a data deletion apparatus according to an embodiment of the present disclosure.
  • the data deletion apparatus is applied to a management server in a distributed storage system, and includes a first determining module 701 and a first sending module 702, where
  • the first determining module 701 is configured to determine a target time point according to a preset data deletion rule when determining that the data needs to be deleted in batches;
  • the first sending module 702 is configured to send the determined target time point to the storage server, so that the storage server that receives the target time point deletes the data to be deleted according to the target time point, where the data to be deleted is the storage time at the target time.
  • the stored data before the point is configured to send the determined target time point to the storage server, so that the storage server that receives the target time point deletes the data to be deleted according to the target time point, where the data to be deleted is the storage time at the target time. The stored data before the point.
  • the management server after determining the target time point, sends the determined target time point to the storage server, so that the storage server can delete the data to be deleted according to the target time point, specifically, storing the data before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes a data block, which can reduce the network interaction between the management server and the storage server. The time spent, reducing the time that the storage server waits for instructions, increases the speed of data deletion, and thus improves the efficiency of storage space release.
  • the first obtaining module is configured to obtain a standard time according to a system time of each storage server, where the standard time is a synchronization process performed on a system time of each storage server. time;
  • the storage time is a standard time when the storage server stores data
  • the target time point is the latest storage time of the storage time of the data to be deleted. .
  • the first obtaining module includes:
  • a judging sub-module configured to determine whether the number of all system times obtained is greater than a preset number
  • a first determining sub-module configured to sort all the system times except a maximum system time and a minimum system time according to the size, if the judgment result of the sub-module is yes, and rank the order in the middle System time, as standard time,
  • a calculation sub-module that calculates the average of all system times obtained as standard time.
  • the device may further include:
  • a saving module for saving the standard time obtained each time
  • the first obtaining module is used to obtain the saved standard time when restarting.
  • the system time of one storage server may be synchronized with the system time of other servers by using the network time protocol NTP.
  • the first determining module is a partition target time point determining module
  • the partition target time point determining module is configured to determine, according to a preset data deletion rule of each partition, a target time point of each first partition to which data is to be deleted;
  • the device also includes:
  • Send module specifically for:
  • the target time point and the corresponding partition identifier delete the data to be deleted stored in the corresponding first partition.
  • a partition target time point determining module includes:
  • a second determining submodule configured, for each first partition, a difference between the obtained current time and a preset storage period of the first partition as a target time point of the first partition.
  • a partition target time point determining module includes:
  • a third determining submodule configured to determine, according to a current storage amount of each partition and a storage threshold preset for each partition, a storage amount to be deleted of each partition;
  • a fourth determining submodule configured, for each first partition that is not to be deleted, according to a pre-recorded data size of the first partition stored data, according to the pre-recorded data stored in the first partition Determining the data to be deleted in the first partition until the difference between the determined total amount of data of the data to be deleted and the amount of the to-be-deleted storage of the first partition is within a preset data range;
  • the device further includes:
  • the first deleting module is configured to delete index information of data stored before the target time point.
  • the first deleting module is specifically configured to:
  • index information for data stored by the first partition before the target time point of the first partition is deleted.
  • the system time of the management server is synchronized with the system time of the NTP server by using the NTP.
  • FIG. 8 is a schematic structural diagram of another data deletion apparatus according to an embodiment of the present disclosure.
  • the data deletion apparatus is applied to a storage server in a distributed storage system, and includes a receiving module 801, a second determining module 802, and a second deleting module 803. among them,
  • the receiving module 801 is configured to receive a target time point sent by the management server, where the target time point is determined according to a preset data deletion rule when the management server determines that the data needs to be deleted in batches;
  • a second determining module 802 configured to determine data to be deleted according to a target time point
  • the second deleting module 803 is configured to delete the determined data to be deleted.
  • the storage server may determine the data to be deleted according to the target time point, and delete and delete the determined data to be deleted, specifically, the data stored before the target time point is deleted, that is, multiple data may be deleted.
  • the block can reduce the time consumed by the network interaction between the management server and the storage server, and reduce the time for the storage server to wait for the instruction, compared with the method in which the storage server receives a delete instruction and deletes a data block. Increased data deletion speed, which in turn increases the efficiency of storage space release.
  • the receiving module 801 is further configured to receive a partition identifier of the first partition.
  • the second determining module 802 is configured to determine, according to the partition identifier of the first partition, whether the storage server stores data corresponding to the first partition for each first partition; if yes, the first partition is The data stored before the target time point of the first partition is determined as the data to be deleted of the first partition;
  • the second deleting module 803 is specifically configured to delete the to-be-deleted data of each determined first partition.
  • the device may further include:
  • a second sending module configured to periodically send a system time of the storage server itself to the management server, so that the management server periodically obtains a standard time according to a system time of each storage server, where the standard time is for each storage The time after the server's system time is synchronized.
  • the system time of the storage server can be synchronized with the system time of the NTP server by using NTP.
  • the embodiment of the present application further provides a management server, as shown in FIG. 9, including a first processor 901, a first communication interface 902, a first memory 903, and a first communication bus 904, wherein the first processor 901, The first communication interface 902, the first memory 903 completes communication with each other through the first communication bus 904,
  • a first memory 903, configured to store a computer program
  • the target time point is determined according to a preset data deletion rule
  • the first communication bus mentioned by the above management server may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.
  • the first communication interface is used for communication between the above management server and other devices.
  • the first memory may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. Alternatively, the memory may also be at least one storage device located remotely from the aforementioned processor.
  • RAM random access memory
  • non-volatile memory such as at least one disk storage.
  • the memory may also be at least one storage device located remotely from the aforementioned processor.
  • the first processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP processor, etc.), or a digital signal processor (Digital Signal Processing, referred to as DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.
  • CPU central processing unit
  • NP processor network processor
  • DSP digital signal processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the management server after determining the target time point, sends the determined target time point to the storage server, so that the storage server can delete the data to be deleted according to the target time point, specifically, storing the data before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes a data block, which can reduce the network interaction between the management server and the storage server. The time spent, reducing the time that the storage server waits for instructions, increases the speed of data deletion, and thus improves the efficiency of storage space release.
  • a computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the application of any of the foregoing embodiments is implemented.
  • the data deletion method for the management server is provided.
  • the management server after determining the target time point, sends the determined target time point to the storage server, so that the storage server can delete the data to be deleted according to the target time point, specifically, storing the data before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • the storage server receives a delete instruction and deletes a data block, which can reduce the network interaction between the management server and the storage server. The time spent, reducing the time that the storage server waits for instructions, increases the speed of data deletion, and thus improves the efficiency of storage space release.
  • the embodiment of the present application further provides a storage server, as shown in FIG. 10, including a second processor 1001, a second communication interface 1002, a second memory 1003, and a second communication bus 1004, wherein the second processor 1001, The second communication interface 1002, the second memory 1003 completes communication with each other through the second communication bus 1004.
  • the storage server may determine the data to be deleted according to the target time point, and delete and delete the determined data to be deleted, specifically, the data stored before the target time point is deleted, that is, multiple data may be deleted.
  • the block can reduce the time consumed by the network interaction between the management server and the storage server, and reduce the time for the storage server to wait for the instruction, compared with the method in which the storage server receives a delete instruction and deletes a data block. Increased data deletion speed, which in turn increases the efficiency of storage space release.
  • a computer readable storage medium stores a computer program, and when the computer program is executed by the processor, the application of any of the foregoing embodiments is implemented. Data deletion method for the storage server.
  • the storage server may determine the data to be deleted according to the target time point, and delete and delete the determined data to be deleted, specifically, the data stored before the target time point is deleted, that is, multiple data may be deleted.
  • Block compared to the manner in which the storage server receives a delete instruction in the related art, deleting one data block can reduce the time consumed by the network interaction between the management server and the storage server, and reduce the waiting time of the storage server waiting for the instruction. Increased data deletion speed, which in turn increases the efficiency of storage space release.
  • an application for executing a data deletion method applied to a management server according to any one of the above embodiments at runtime is further provided.
  • the management server when the data is deleted by running the application, sends the determined target time point to the storage server after determining the target time point, so that the storage server can delete the to-be deleted according to the target time point.
  • Data specifically, the data stored before the target time point is deleted, that is, multiple data blocks can be deleted, which can be reduced in the manner that the storage server receives a delete instruction and deletes one data block in the related art.
  • the time spent managing the network interaction between the server and the storage server reduces the time that the storage server waits for instructions, increases the speed of data deletion, and improves the efficiency of storage space release.
  • an application is further provided for executing a data deletion method applied to a storage server according to any one of the above embodiments at runtime.
  • the storage server may determine the data to be deleted according to the target time point, and delete the determined data to be deleted, specifically, the data to be stored before the target time point.
  • the data is deleted, that is, multiple data blocks can be deleted.
  • deleting one data block can reduce the network interaction between the management server and the storage server and consume Time, reducing the time that the storage server waits for instructions, increasing the speed of data deletion, thereby increasing the efficiency of storage space release.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请实施例提供了一种数据删除方法及分布式存储***,应用于分布式存储***中的管理服务器的方法,包括:当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间点删除待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。应用本申请实施例提供的数据删除方法,可以提高存储空间释放的效率。

Description

数据删除方法及分布式存储***
本申请要求于2017年9月25日提交中国专利局、申请号为201710876800.3、发明名称为“数据删除方法及分布式存储***”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域,特别是涉及一种数据删除方法及分布式存储***。
背景技术
分布式存储***是将不同的服务器通过网络进行互联并协同工作,以向用户提供海量数据存储功能的***。分布式存储***主要包括:管理服务器和存储服务器,管理服务器用于存储的数据的索引信息,数据的索引信息包括数据的大小、存储时间以及存储位置等信息,在数据分块存储的情况下,数据的索引信息还包括该数据对应的数据块的索引信息;示例性的,管理服务器可以为对象存储***中的元数据服务器((Metadata Server,MDS);存储服务器主要用于存储数据,示例性的,存储服务器可以为对象存储服务器(Object-based Storage Device,OSD)。
由于分布式存储***的存储空间不可能无限扩展,随着用户的存储需求越来越多,需要存储的数据越来越多,为了满足用户的存储需求,可以批量删除部分已经存储的数据,释放存储空间,使得分布式存储***能够存储更多的新的数据。
相关技术中的数据删除方法为:管理服务器定期根据预设的存储周期和数据的存储时间,确定过期的数据;针对所确定的过期的数据中的每一数据,确定存储有该数据的存储服务器,并针对该数据的每一数据块生成一个删除指令;向所确定的存储服务器发送删除指令以使接收到删除指令的存储服务器删除该删除指令对应的数据块。管理服务器可以在向存储服务器发送删除指令后删除所确定的数据的索引信息。
在上述方法中,管理服务器针对一个数据块生成一个删除指令,并通过网络协议将删除指令发送给存储服务器,通过网络协议进行指令的传输,这 种网络交互需要一定的时间,当存在大量的删除指令需要发送的情况下,会产生大量的网络交互,这些网络交互会消耗大量的时间,从而会导致批量删除数据时速度比较慢,存储空间释放的效率不高。
发明内容
本申请实施例的目的在于提供一种数据删除方法及分布式存储***,以实现提高存储空间释放的效率。具体技术方案如下:
第一方面,为了达到上述目的,本申请实施例提供了一种数据删除方法,应用于分布式存储***中的管理服务器,所述方法包括:
当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;
将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间点删除待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
可选的,所述方法还包括:
定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
所述存储时间为存储服务器存储数据时的标准时间;
所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
可选的,所述定时根据各个存储服务器的***时间,获得标准时间,包括:
定时获取各个存储服务器的***时间;
判断所获得的所有***时间的个数是否大于预设数量;
如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
或,
计算除一个最大***时间和一个最小***时间之外的其他所有***时间 的平均值,作为标准时间;
如果否,计算所获得的所有***时间的平均值,作为标准时间。
可选的,所述根据预设的数据删除规则,确定目标时间点,包括:
根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;
在所述将所确定的目标时间点发送给存储服务器之前,所述方法还包括:
获取每个第一分区的分区标识;
所述将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间点删除待删除数据,包括:
将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应第一分区中所存储的待删除数据。
可选的,所述根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
获得当前时间;
针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
可选的,所述根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;
针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;
针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
可选的,在根据预设的数据删除规则,确定目标时间点之后,所述方法还包括:
删除在所述目标时间点之前存储的数据的索引信息。
可选的,所述删除在所述目标时间点存储的数据的索引信息,包括:
针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
第二方面,为达到上述目的,本申请实施例还提供了一种数据删除方法,应用于分布式存储***中的存储服务器,所述方法包括:
接收管理服务器发送的目标时间点,其中,所述目标时间点是所述管理服务器确定需要批量删除数据时,根据预设的数据删除规则确定的;
根据目标时间点确定待删除数据;
删除所确定的待删除数据。
第三方面,为了达到上述目的,本申请实施例还提供了一种数据删除方法,应用于分布式存储***,所述分布式存储***包括管理服务器和存储服务器,所述方法包括:
所述管理服务器当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器;
所述存储服务器在将接收到目标时间点后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
第四方面,为了达到上述目的,本申请实施例公开了一种分布式存储***,所述***包括管理服务器和存储服务器,
所述管理服务器,用于当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器;
所述存储服务器,用于在接收到所述目标时间点后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
又一方面,为了达到上述目的,本申请实施例还提供了一种管理服务器,包括:第一处理器、第一通信接口、第一存储器和第一通信总线,其中,所述第一处理器、所述第一通信接口、所述第一存储器通过所述第一通信总线完成相互间的通信;
所述第一存储器,用于存放计算机程序;
所述第一处理器,用于执行所述第一存储器上所存放的计算机程序,实现上述应用于管理服务器的任一数据删除方法步骤。
又一方面,为了达到上述目的,本申请实施例还提供了一种存储服务器,包括:第二处理器、第二通信接口、第二存储器和第二通信总线,其中,所述第二处理器、所述第二通信接口、所述第二存储器通过所述第二通信总线完成相互间的通信;
所述第二存储器,用于存放计算机程序;
所述第二处理器,用于执行所述第二存储器上所存放的计算机程序,实现上述应用于存储服务器的任一数据删除方法步骤。
又一方面,为了达到上述目的,本申请实施例还提供了一种计算机可读存储介质,其特征在于,所述存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现上述应用于管理服务器的任一数据删除方法步骤。
又一方面,为了达到上述目的,本申请实施例还提供了一种计算机可读存储介质,其特征在于,所述存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现上述应用于存储服务器的任一数据删除方法步骤。
又一方面,为了达到上述目的,本申请实施例还提供了一种应用程序,其特征在于,所述应用程序用于在运行时执行上述应用于管理服务器的任一数据删除方法步骤。
又一方面,为了达到上述目的,本申请实施例还提供了一种应用程序, 其特征在于,所述应用程序用于在运行时执行上述应用于存储服务器的任一数据删除方法步骤。
本申请实施例提供的数据删除方法及分布式存储***,管理服务器可以向存储服务器发送目标时间点,使得存储服务器可以根据该目标时间点删除待存储数据,不需要针对每个数据块发送删除指令,减少了大量的网络交互,因此减少了管理服务器与存储服务器之间的网络交互而消耗的时间,提高存储空间释放的效率。当然,实施本申请的任一产品或方法必不一定需要同时达到以上所述的所有优点。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的数据删除方法的一种流程示意图;
图2为本申请实施例提供的数据删除方法的又一种流程示意图;
图3为本申请实施例提供的数据删除方法的另一种流程示意图;
图4为本申请实施例提供的另一种数据删除方法的流程示意图;
图5为本申请实施例提供的另一种数据删除方法的流程示意图;
图6本申请实施例提供的分布式存储***的结构示意图;
图7为本申请实施例提供的数据删除装置的结构示意图;
图8为本申请实施例提供的另一种数据删除装置的结构示意图;
图9为本申请实施例提供的管理服务器的结构示意图;
图10为本申请实施例提供的存储服务器的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而 不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
为了解决相关技术中批量删除数据时速度比较慢,存储空间释放的效率不高的技术问题,本申请实施例提供了一种数据删除方法及分布式存储***。
下面首先对本申请实施例提供的一种数据删除方法进行详细说明。
本申请实施例提供的数据删除方法可以优选应用于分布式存储***中的管理服务器,也可以优选应用于分布式存储***中的存储服务器,还可以优选应用于分布式存储***。分布式存储***包括管理服务器和存储服务器,管理服务器主要用于存储数据的索引信息的服务器,存储服务器主要用于存储数据。在本实施例中,管理服务器当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点,并将所确定的目标时间点发送给存储服务器,存储服务器按照所述目标时间点删除待删除数据。管理服务器不需要针对每个数据块发送删除指令,减少了大量的网络交互,因此减少了管理服务器与存储服务器之间的网络交互而消耗的时间,提高存储空间释放的效率。在本申请实施例中,分布式存储***可以为对象存储***,对象存储***可以为用户提供海量、安全、高可靠和易扩展的云存储服务,当分布式存储***为对象存储***时,管理服务器可以为MDS,存储服务器可以为OSD。
图1为本申请实施例提供的数据删除方法的一种流程示意图,应用于分布式存储***中的管理服务器,方法包括:
S101:当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点。
本实施例中,可以有如下2种方式来确定是否需要批量删除数据:
第一种:按照预设时间间隔,当距离上一次确定是否需要批量删除数据达到预设时间时间间隔时,确定是否需要批量删除数据。
预设时间间隔可以基于在预设时长内待存储的数据量和存储服务器的存储容量确定,也可以根据存储的数据的类型确定,当然,也可以根据其他的方式确定,本申请实施例在此并不进行限定。示例性的,如果存储的数据是监控视频,预设时长内产生了海量的视频数据,可以将时间间隔设置的短一 些;如果存储的用户的购物数据,因为每一个购物数据所占用的存储空间都很小,可以将时间间隔的可以设置的长一些;在存储的视频数据,有购物数据,也有其他类型的数据的情况下,可以根据实际情况设置时间间隔。
第二种:管理服务器收到用户发送的指令。
这里所说的指令是批量删除指令,管理服务器在收到指令后,会根据预设的数据删除规则,确定目标时间点。
数据删除规则可以是预先根据实际情况设置,具体的,可以基于预先设置的存储周期确定,也可以基于预设的存储阈值确定。
分布式存储***中,数据还可以分区进行存储,在数据分区进行存储的情况下,需要分区的目标时间点,则根据预设的数据删除规则,确定目标时间点,可以包括:
根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点。
这里所说的分区是逻辑上的分区,可以是针对用户的分区,一个分区与一个用户对应,一个用户可以对应多个分区。属于同一个分区的数据可以存储在不同的存储服务器中。针对不同的分区可以设置不同的数据删除规则,也可以设置相同的数据删规则。
第一分区是指待删除数据量不为零的分区,如果一个分区的待删除数据量为零,说明该分区中不存在需要删除的数据,进一步说明根据该分区的数据删除,无法确定目标时间点。
因为第一分区可能存在很多个,对应的,第一分区的目标时间点也有很多,为了使存储服务器能够正确地确定分区与分区的目标时间点之间的关系,需要获取第一分区的分区标识。
分区标识可以是分区的名称,也可以为分区的编号,具体的分区标识可以包括文字、数字、字母、字符中的至少一种。分区标识预先存储在数据库中,也可以存储在管理服务器的内存中。
S102:将所确定的目标时间点发送给存储服务器,以使接收到所述目标 时间点的存储服务器按照所述目标时间点删除待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
本实施例中,如果管理服务器是元数据服务器,还可以在确定目标时间点后,向存储服务器发送之前,确定待删除数据,并删除待删除的索引信息。
存储服务器接收到目标时间点时间后,按照目标时间点,确定待删除数据,并删除所有确定出的待删除数据。存储服务器确定待删除数据的方法可以为:判断自身所记录的数据的存储时间中,是否存在不晚于目标时间点的存储时间,如果存在,将不晚于目标时间点的存储时间对应的数据,确定为待删除数据。
在获取第一分区的目标时间点和分区标识后,可以将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应的第一分区中所存储的待删除数据。
存储服务器在接收到第一分区的目标时间点及对应的分区标识后,确定分区标识所对应的第一分区,根据预先记录的所确定的第一分区的已存储数据的存储时间,确定在该第一分区的目标时间点之前存储的数据,作为该第一分区对应的待删除数据,删除所确定的该第一分区对应的待删除数据。这里所说的对应的分区标识,是指所发送的第一分区的目标时间点对应的第一分区的分区标识。
当第一分区只有一个时,可以将该第一分区的目标时间点和该第一分区的分区标识分别发送给存储服务器,也可以将该第一分区的目标时间点和该第一分区的分区标识作为一个整体发送给存储服务器。当第一分区有多个时,为了避免发生混乱,可以将一个第一分区的目标时间点与该第一分区的分区标识作为一个整体发送给存储服务器。
在相关技术中,存储服务器接收一个删除指令,删除已存储的与删除指令对应的数据块,存储服务器与管理服务器之间会产生大量的网络交互,这些网络交互会消耗大量的时间,导致数据删除速度比较慢。如果数据的存储 速度很快,数据的存储速度很有可能大于数据的删除速度。在这种情况下,存储服务器的存储空间不断地被占用,最终会导致存储空间不足,存储空间不足,会使得后续的待存储数据存储失败。在本申请实施例中,管理服务器通过发送目标时间点给存储服务器,存储服务器删除在目标时间点之前存储的数据,以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度。相较于相关技术,在相同的存储速度下,不会发生空间不足的情况,可以尽可能地存储全部的待存储数据。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,使得存储服务器可以根据目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率,进而提高存储空间释放的效率。
在本申请实施例的一种具体实施方式中,目标时间点可以根据一个与存储服务器***时间同步的标准时间来确定。例如,可以采用如下步骤来确定标准时间:
定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
所述存储时间为存储服务器存储数据时的标准时间;
所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
在本申请实施例中,定时的时长可以与上述所说的预设的时间间隔可以相同,也可以不同,在这里不进行限定,具体的定时的时长可以根据实际情况而定。
存储服务器的***时间是指存储服务器当前的***时间,可以是存储服务器发送给管理服务器的,也可以是其他服务器收集存储服务器的***时间,并将所搜集的***时间发送给管理服务器。
存储服务器将自身的***时间上报给管理服务器,管理服务器立即将所获得的***时间保存到内存中,并对各个存储服务器的***时间进行同步处理,获得标准时间,标准时间是对存储服务器***时间进行同步处理后的时间,这里所说的标准时间与存储服务器的***时间之间的差值在预设时间阈值内,在一般情况下,标准时间与大多数存储服务器的***时间的差值在秒级。
管理服务器在接收到存储服务器的***时间后,在一种实施方式中,可以通过以下步骤对各个存储服务器的***时间进行同步处理,获得标准时间:
定时获取各个存储服务器的***时间;
判断所获得的所有***时间的个数是否大于预设数量;
如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
或,
计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
如果否,计算所获得的所有***时间的平均值,作为标准时间。
存储服务器的***时间有可能会跑偏,在获取多个存储服务器的***时间的情况下,需要剔除跑偏的***时间,避免影响到所获得的标准时间的准确性,具体的做法可以为:在判断出所获取的所有***时间的个数大于预设数量的情况下,剔除一个最大的***时间和一个最小的***时间,按照大小对剩下的***时间进行排序,将排在中间的***时间,作为标准时间,这样既可以避免跑偏的***时间影响所获得的目标时间点的准确性,同时也可以选择一个比较接近于大多数存储服务器的***时间,可以使得所获得的标准时间与绝大多数存储服务器的***时间相差在预设的时间范围内。当然,也可以计算剩下的***时间的平均值,作为准时间,这样也可以避免跑偏的***时间影响所获得的目标时间点的准确性。
其中,上述最大的***时间即为最早的***时间;最小的***时间即为 最晚的***时间。在剔除一个最大的***时间和一个最小的***时间后,剩余***时间的数量为奇数时,上述排在中间的***时间即为排序排在最中间的***时间。在剔除一个最大的***时间和一个最小的***时间后,剩余***时间的数量为偶数时,上述排在中间的***时间可以为排序排在最中间的两个***时间中的任一个,也可以为排序排在最中间的两个***时间的平均值。
在所获得的***时间的数量不大于预设数量的情况下,可以直接将所获得的所有***时间的平均值,作为标准时间。
在本发明申请实施例中,存储服务器有可能出现故障导致存储服务器不在线,所获取的是在线存储服务器的***时间,并且所获得的***时间的数量可能是变化的,是在获取存储服务器的***时间时在线存储服务器的数量。
在获得标准时间后,可以将保存每次获得的标准时间,管理服务器重启时获得所保存的标准时间。
保存所获得的标准时间,具体的,可以将获得的标准时间保存到数据库中,数据库(Data Base,DB)是指在计算机的存储设备上合理存放的相关联的有结构的数据集合,一个数据库含有各种部分,包括表、视图、存储过程、记录、字段、索引等。其中,存储过程可以包括存储方式、数据标识等。
保存所获得的存储服务器的标准时间也可以方便管理服务器重启时读取,在管理服务器重启时,有可能不能立即获得存储服务器的***时间,从而获得标准时间,但有可能接收到待存储数据,此时可以获得所保存的存储服务器的标准时间,将当前所保存的标准时间作为待存储数据的存储时间,尽量保证管理服务器所记载的数据的存储时间与存储服务器记载的该数据的存储时间一致。
获得标准时间的主要目的是为了使得管理服务器保存的数据的索引信息中记载的数据的存储时间和存储服务器记录的该数据的存储时间一致。为了实现这一目的,可以在保存根据各个存储服务器的***时间获得标准时间之后,按照预设的时间步长,在当前所保存的标准时间上增加时间步长,作为当前的标准时间,管理服务器在保存数据的索引信息时,将该数据的存储时 间记载为当前的标准时间。
为了获得比较准确的标准时间,需要存储服务器的***时间同步,存储服务器***时间同步的方式为:
各个存储服务器的***时间均可以通过网络时间协议NTP与NTP服务器的***时间同步。
NTP(Network Time Protocol)是用来使计算机时间同步化的一种协议,可以使计算机对其服务器或时钟源(如石英钟,GPS(Global Positioning System,全球定位***)等等)做同步化,从而提供高精准度的时间校正。具体的,可以是存储服务器与NTP服务器建立通信连接,将自身的***时间与NTP服务器的***时间保持同步。这里所说的NTP服务器可以为专业的NTP服务器,示例性的,可以是国家授时中心服务器的NTP服务器;也可以为分布式存储***中专门设置的服务器。
下面以一个具体实例说明时间同步的过程:
服务器A为存储服务器,未与NTP服务器的***时间同步之前,服务器A的***时间为上午10:00:00,NTP服务器的***时间为上午11:00:00。则时间同步的具体工作过程为:
1、服务器A发送一个NTP消息包给NTP服务器,该NTP消息包携带有该NTP消息包离开服务器A时的时间戳,该时间戳为上午10:00:00(t1)。
2、当此NTP消息包到达NTP服务器时,NTP服务器加上自己的时间戳,该时间戳为上午11:00:01(t2)。
3、当此NTP消息包离开NTP服务器时,NTP服务器再加上自己的时间戳,该时间戳为上午11:00:02(t3)。
4、当服务器A接收到该响应消息包时,加上一个新的时间戳,该时间戳为上午10:00:03(t4)。
服务器A通过计算可以得到:NTP消息包来回一个周期的时延=(t4-t1)-(t3-t2)。服务器A相较于NTP服务器的时间差=((t2-t1)+(t3-t4))/2。服务器A利用这些信息,可以将自身的***时间设置为11:00:03。每一个存储服务 器通过以上方式与NTP服务器的***时间保持同步,则一个存储服务器的***时间与其他存储服务器的***时间保持同步。
可见,本实施例中获得对各个存储服务器的***时间进行同步处理后的标准时间,能够使得管理服务器记录索引时所使用的存储时间与存储服务器存储数据时所记录的存储时间同步,从而进一步保证删除数据的准确性。
在本申请实施例的另一种具体实施方式中,目标时间点可以根据管理服务器的***时间来确定,这种具体实施方式中,为了保证记录索引信息所用到的存储时间与存储数据时所记录的存储时间同步,可以使管理服务器和存储服务器分别通过网络时间协议NTP与NTP服务器的***时间同步,管理服务器的***时间与NTP服务器的***时间同步的方式可以与存储服务器的***时间跟NTP服务器的***时间同步的方式相同,这里不再赘述。
可见,本实施例中管理服务器和存储服务器均与NTP服务器同步了***时间,也能够进一步保证管理服务器记录索引信息时所使用的存储时间与存储服务器存储数据时所记录的存储时间同步,从而进一步保证删除数据的准确性。
管理服务器会存储每一个数据的索引信息,当存储服务器将数据删除之后,为了节省管理服务器的存储空间,也为了在用户请求读取已经删除的数据时,减少用户的等待时间,可以删除在目标时间点之前存储的数据的索引信息。因为索引信息中包括数据的最后修改时间,即数据的存储时间,可以将数据的存储时间在目标时间点之前的数据对应的索引信息确定为待删除索引信息,进而删除待删除索引信息。
作为本申请实施例的一种实施方式,删除在目标时间点之前存储的数据的索引信息,包括:
针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
第一分区是待删除数据量不为零的分区,说明需要删除第一分区已存储的待删除数据,也需要删除第一分区已存储的待删除数据的索引信息。管理 服务器从针对第一分区存储的数据的索引信息中,确定存储时间在该第一分区的目标时间点之前的数据的索引信息,作为该第一分区的待删除索引信息,并删除该第一分区的待删除索引信息。
为了能够将待删除数据和待删除数据的索引信息均删除,需要保持管理服务器存储的数据的索引信息中记载的存储时间和存储服务器记载的该数据的存储时间一致,进一步地,是保证记录索引信息所使用的时间与存储服务器记录数据的存储时间所使用的时间一致。在本申请实施例中,记录索引信息所使用的时间可以是当前所保存的标准时间,存储服务器记录数据的存储时间所使用的时间可以是存储服务器的***时间,存储服务器的***时间与NTP服务器的***时间保持同步;另外在管理服务器的***时间、存储服务器的***时间均与NTP服务器的***时间保持同步的情况下,记录索引信息所使用的时间可以是管理服务器的***时间,存储服务器记录数据的存储时间所使用的时间可以是存储服务器的***时间。
可见,管理服务器删除在目标时间点之前存储的索引信息,不仅可以节省管理服务器的存储空间。由于管理服务器和的存储服务器并发执行删除操作,可以避免请求等待,极大的提高存储空间释放的效率,避免磁盘空间释放不及时造成的空间释放慢而导致的最新待存储数据丢失的问题。
下面结合另一具体实施例,对本申请实施例所提供的一种数据删除方法进行介绍。
图2为本申请实施例提供的数据删除方法的又一种流程示意图,方法包括:
S201:当确定需要批量删除数据时,获得当前时间。
当前时间可以是管理服务器的***时间,也可以是数据库中当前存储的标准时间。数据库中当前存储的标准时间可以为基于存储服务器的***时间所获得的标准时间,也可以是在基于存储服务器的***时间所获得的标准时间的基础上,每隔预设的时间步长增加该时间步长所得到的。这里所说的管理服务器的***时间可以是与存储服务器的***时间同步后的***时间。
S202:针对每个第一分区,将所获得的当期时间与该第一分区预设的存 储周期之差,作为该第一分区的目标时间点。
分区的存储周期是根据用户的需求设置的,不同的分区可以设置相同的存储周期,也可以设置不同的存储周期。在获得当前时间和第一分区的存储周期之后,获得该第一分区的目标时间点。示例性的,当前时间为2017年8月15日上午9:00,第一分区为分区A,分区A的预设的存储周期为10天,则计算出的分区A的目标时间点为2017年8月5日上午9:00。需要说明的是,在本申请实施例中,将当前时间与第一分区的存储周期之差,作为该第一分区的目标时间点可以理解为是通过周期覆盖逻辑原理确定目标时间点。
S203:获取每个第一分区的分区标识。
S204:将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应第一分区中所存储的待删除数据。
本申请实施例中,管理服务器通过周期覆盖原理确定第一分区的目标时间点,将所确定的第一分区目标时间点和对应的分区标识发送给存储服务器,使得存储服务器可以删除第一分区中存储的待删除数据,一次可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,本申请实施例中,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
下面结合另一具体实施例,对本申请实施例所提供的一种数据删除方法进行介绍。
图3为本申请实施例提供的数据删除方法的另一种流程示意图,方法包括:
S301:当确定需要批量删除数据时,根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量。
管理服务器中会保存每一分区存储的数据的索引信息,索引信息中包含 每一数据的数据大小,基于所保存的索引信息,可以计算该分区当前的存储量,当然,管理服务器也可以记录有分区当前的存储量,当一个数据存储至该分区后,管理服务器在记录的该分区的存储量上增加该数据的大小。
每一分区的存储阈值是预先根据实际情况设置的,存储阈值可以是一个固定的值,也可以是根据分区的存储总量和该分区预设的存储比例阈值确定的,示例性的,预先可以将分区B的存储阈值设置为15G,分区B当前的存储量为20G,则可以确定分区B的待删除存储量为5G;分区C的存储总量为15G,预设的存储比例阈值为80%,则分区C的存储阈值为12G,进而确定分区C的待删除存储量为3G。
S302:针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内。
如果一个分区待删除存储量为零,说明当前该分区不存在需要删除的数据,则该分区不能称之为第一分区。
针对一个第一分区,管理服务器按照该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,具体的实现方式可以为:将存储时间最早的数据确定为待删除数据,判断所有待删除的数量总量是否与该第一分区的待删除存储量之差在预设数据范围内,该预设的数据范围针对不同的分区可以相同,也可以不同,针对同一个分区,预设数据范围也可以不同;如果否,则将除已经确定为待删除数据之外的存储的其他数据中,存储时间最早的数据也确定为待删除数据,执行判断所有待删除的数量总量是否与该第一分区的待删除存储量之差在预设数据范围内的在步骤;如果是,则执行S303。
所确定的待删除数据的数据总量有很大可能性不等于该第一分区的待删除数据量,因此,需要满足以下条件即可不继续确定待删除数据:
所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内。
S303:针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
针对每个第一分区所确定待删除数据均是按照存储时间先后顺序确定的,选择最晚的存储时间作为该第一分区的目标时间点,可以理解为将最后确定的待存储数据的存储时间,作为该第一分区的目标时间点。当然,也可以理解为将所有待删除数据的存储时间按时间先后进行排序,选择最晚的存储时间作为该第一分区的目标时间点。
上面所说的确定第一分区的目标时间点是利用容量覆盖逻辑确定的,容量覆盖是根据设置的容量阈值,把最早存储的数据删除,以达到容量符合设置要求的目的。
S304:获取每个第一分区的分区标识。
S305:将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应第一分区中所存储的待删除数据。
本申请实施例中,管理服务器通过容量覆盖原理确定第一分区的目标时间点,将所确定的第一分区目标时间点和对应的分区标识发送给存储服务器,使得存储服务器可以删除第一分区中存储的待删除数据,一次可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,本申请实施例中,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
图4为本申请实施例提供的另一种数据删除方法的流程示意图,应用于分布式存储***中的存储服务器,方法包括:
S401:接收管理服务器发送的目标时间点,其中,所述目标时间点是所述管理服务器确定需要批量删除数据时,根据预设的数据删除规则确定的。
在数据是分区存储的情况下,接收管理服务器发送的目标时间点,可以包括:
接收管理服务器发送的第一分区的目标时间点。
存储服务器在接收第一分区的目标时间点的同时,还会接收第一分区的分区标识。
S402:根据目标时间点确定待删除数据。
存储服务器在接收到目标时间点后,可以根据自身记录的每一数据的存储时间,确定是否存在在目标时间点时间之前存储的数据,如果存在,将在目标时间点之前存储的数据确定为待删除数据。
当数据是分区存储的情况下,根据目标时间点确定待删除数据,可以包括:
针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;
如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据。
一个第一分区的数据可以存储在一个存储服务器中,也可以存储在多个存储服务器中,同样的,一个存储服务器可以存储一个分区的数据,也可以存储多个分区的数据。当存储服务器接收到第一分区的目标时间点和对应的分区标识后,先根据第一分区的分区标识,判断自身是否存储有该第一分区对应的数据,如果有,将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据。示例性的,第一分区为分区1,分区1分区标识为A1,存储服务器记录的分区标识中包括A1,则确定存储服务器存储有分区1对应的数据,将分区1已存储的数据中,在分区1的目标时间点之前存储的数据确定为分区1的待删除数据。
S403:删除所确定的待删除数据。
作为本申请实施例的一种实施方式,删除所确定的待删除数据,包括:
删除所确定的每一第一分区的待删除数据。
本申请实施例中,存储服务器可以根据目标时间点确定待删除数据,并删除所确定的待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
作为本申请实施例的一种实施方式,方法还可以包括:
定时向管理服务器发送所述存储服务器自身的***时间,以使所述管理服务器定时根据各个存储服务器的***时间获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间。
作为本申请实施例的一种实施方式,存储服务器的***时间通过NTP与NTP服务器的***时间同步。
在本申请实施例中,每一存储服务器的***时间均可以通过NTP与NTP服务器的***时间同步,同步的方法与上述所说的同步方法相同,在这里不进行赘述。
图5为本申请实施例提供的又一种数据删除方法的流程示意图,该数据删除方法应用于分布式存储***,分布式存储***包括管理服务器和存储服务器,方法包括:
S501:所述管理服务器当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器。
作为本申请的一种实施方式,管理服务器根据预设的数据删除规则,确定目标时间点,包括:
根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;
所述管理服务器在所述将所确定的目标时间点发送给存储服务器之前,获取每个第一分区的分区标识;
所述管理服务器将所确定的目标时间发送给存储服务器,包括:
将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器。
作为本申请的一种实施方式,所述管理服务器根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
获得当前时间;
针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
作为本申请的一种实施方式,所述管理服务器根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;
针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;
针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
作为本申请的一种实施方式,所述管理服务器删除在所述目标时间点之前存储的数据的索引信息。
作为本申请的一种实施方式,所述管理服务器删除在所述目标时间点之前存储的数据的索引信息,包括:
针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
S502:存储服务器在将接收到目标时间后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间 在所述目标时间点之前的已存储数据。
作为本申请的一种实施方式,所述存储服务器接收所述管理服务器发送的第一分区的目标时间点和对应的分区标识;
所述存储服务器根据目标时间点确定待删除数据,包括:
针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;
如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;
所述存储服务器删除所确定的待删除数据,包括:
删除所确定的每一第一分区的待删除数据。
作为本申请的一种实施方式,所述存储服务器定时向所述管理服务器发送所述存储服务器自身的***时间;
所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
所述存储时间为存储服务器存储数据时的标准时间;
所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
作为本申请的一种实施方式,所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,包括:
定时获取各个存储服务器的***时间;
判断所获得的所有***时间的个数是否大于预设数量;
如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
或,
计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
如果否,计算所获得的所有***时间的平均值,作为标准时间。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,存储服务器可以根据目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
图6为本申请实施例提供的分布式存储***的结构示意图,分布式存储***包括管理服务器610和存储服务器620,其中,
管理服务器610,用于当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器620;
存储服务器620,用于在接收到所述目标时间点后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,存储服务器可以根据该目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
作为本申请的一种实施方式,存储服务器620,用于定时向所述管理服务器610发送所述存储服务器620自身的***时间;
所述管理服务器610,用于定时根据各个存储服务器620的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器620的***时间进行同步处理后的时间;
所述存储时间为存储服务器620存储数据时的标准时间;
所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
作为本申请的一种实施方式,所述管理服务器610定时根据各个存储服务器620的***时间,获得标准时间,包括:
定时获取各个存储服务器620的***时间;
判断所获得的所有***时间的个数是否大于预设数量;
如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
或,
计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
如果否,计算所获得的所有***时间的平均值,作为标准时间。
作为本申请的一种实施方式,所述管理服务器610用于根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;获取每个第一分区的分区标识;将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器620;
所述存储服务器620,用于接收所述管理服务器610发送的第一分区的目标时间点和对应的分区标识;针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器620是否存储有该第一分区对应的数据;如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;删除所确定的每一第一分区的待删除数据。
作为本申请的一种实施方式,所述管理服务器610,用于根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
获得当前时间;
针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
作为本申请的一种实施方式,所述管理服务器610,用于根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
作为本申请的一种实施方式,所述管理服务器610,用于删除在所述目标时间点之前存储的数据的索引信息。
作为本申请的一种实施方式,所述管理服务器610,用于针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
与上述方法实施例相对应,本申请实施例还提供一种数据删除设置。
图7为本申请实施例提供的数据删除装置的结构示意图,数据删除装置应用于分布式存储***中的管理服务器,包括第一确定模块701和第一发送模块702,其中,
第一确定模块701,用于当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;
第一发送模块702,用于将所确定的目标时间点发送给存储服务器,以使接收到目标时间点的存储服务器按照目标时间点删除待删除数据,其中,待删除数据为存储时间在目标时间点之前的已存储数据。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,使得存储服务器可以根据该目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的 时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
作为本申请实施例的一种实施方式,第一获得模块,用于定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
所述存储时间为存储服务器存储数据时的标准时间;
所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。。
作为本申请实施例的一种实施方式,第一获得模块,包括:
获取子模块,用于定时获取各个存储服务器的***时间;
判断子模块,用于判断所获得的所有***时间的个数是否大于预设数量;
第一确定子模块,用于在判断子模块的判断结果为是的情况下,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
或,
计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
计算子模块,用于计算所获得的所有***时间的平均值,作为标准时间。
作为本申请实施例的一种实施方式,该装置还可以包括:
保存模块,用于保存每次获得的标准时间;
第一获得模块,用于重启时获得所保存的标准时间。
作为本申请实施例的一种实施方式,一个存储服务器的***时间可以是通过网络时间协议NTP与其他服务器的***时间同步的。
作为本申请实施例的一种实施方式,第一确定模块为分区目标时间点确定模块,
该分区目标时间点确定模块,用于根据预设的每个分区的数据删除规则, 确定每个要删除数据的第一分区的目标时间点;
装置还包括:
获取模块,用于获取每个第一分区的分区标识;
发送模块,具体用于:
将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应第一分区中所存储的待删除数据。
作为本申请实施例的一种实施方式,分区目标时间点确定模块,包括:
获得子模块,用于获得当前时间;
第二确定子模块,用于针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
作为本申请实施例的一种实施方式,分区目标时间点确定模块,包括:
第三确定子模块,用于根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;
第四确定子模块,用于针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;
选择子模块,用于针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
作为本申请实施例的一种实施方式,装置还包括:
第一删除模块,用于删除在目标时间点之前存储的数据的索引信息。
作为本申请实施例的一种实施方式,第一删除模块,具体用于:
针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
作为本申请实施例的一种实施方式,管理服务器的***时间通过NTP与NTP服务器的***时间同步。
图8为本申请实施例提供的另一种数据删除装置的结构示意图,数据删除装置应用于分布式存储***中的存储服务器,包括接收模块801、第二确定模块802和第二删除模块803,其中,
接收模块801,用于接收管理服务器发送的目标时间点,其中,所述目标时间点是所述管理服务器确定需要批量删除数据时,根据预设的数据删除规则确定的;
第二确定模块802,用于根据目标时间点确定待删除数据;
第二删除模块803,用于删除所确定的待删除数据。
本申请实施例中,存储服务器可以根据该目标时间点确定待删除数据,并删删除所确定的待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
作为本申请实施例的一种实施方式,接收模块801,还用于接收第一分区的分区标识;
第二确定模块802,具体用于针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;
第二删除模块803,具体用于删除所确定的每一第一分区的待删除数据。
作为本申请实施例的一种实施方式,该装置还可以包括:
第二发送模块,用于定时向管理服务器发送所述存储服务器自身的***时间,以使所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间。
作为本申请实施例的一种实施方式,存储服务器的***时间可以通过NTP与NTP服务器的***时间同步。
本申请实施例还提供了一种管理服务器,如图9所示,包括第一处理器901、第一通信接口902、第一存储器903和第一通信总线904,其中,第一处理器901,第一通信接口902,第一存储器903通过第一通信总线904完成相互间的通信,
第一存储器903,用于存放计算机程序;
第一处理器901,用于执行第一存储器903上所存放的程序时,实现如下步骤:
当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;
将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间点删除待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
上述管理服务器提到的第一通信总线可以是外设部件互连标准(Peripheral Component Interconnect,简称PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,简称EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示,图中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。
第一通信接口用于上述管理服务器与其他设备之间的通信。
第一存储器可以包括随机存取存储器(Random Access Memory,简称RAM),也可以包括非易失性存储器(non-volatile memory),例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远离前述处理器的存储装 置。
上述的第一处理器可以是通用处理器,包括中央处理器(Central Processing Unit,简称CPU)、网络处理器(Network Processor,简称NP)等;还可以是数字信号处理器(Digital Signal Processing,简称DSP)、专用集成电路(Application Specific Integrated Circuit,简称ASIC)、现场可编程门阵列(Field-Programmable Gate Array,简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,使得存储服务器可以根据该目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
在本申请的又一实施例中,还提供了一种计算机可读存储介质,计算机可读存储介质内存储有计算机程序,计算机程序被处理器执行时实现上述实施例中任一所述的应用于管理服务器的数据删除方法。
本申请实施例中,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,使得存储服务器可以根据该目标时间点删除待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
本申请实施例还提供了一种存储服务器,如图10所示,包括第二处理器1001、第二通信接口1002、第二存储器1003和第二通信总线1004,其中,第二处理器1001,第二通信接口1002,第二存储器1003通过第二通信总线1004完成相互间的通信,
第二存储器1003,用于存放计算机程序;
第二处理器1001,用于执行第二存储器803上所存放的程序时,实现如下步骤:
接收管理服务器发送的目标时间点,其中,所述目标时间点是所述管理服务器确定需要批量删除数据时,根据预设的数据删除规则确定的;
根据目标时间点确定待删除数据;
删除所确定的待删除数据。
本申请实施例中,存储服务器可以根据该目标时间点确定待删除数据,并删删除所确定的待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
在本申请的又一实施例中,还提供了一种计算机可读存储介质,计算机可读存储介质内存储有计算机程序,计算机程序被处理器执行时实现上述实施例中任一所述的应用于存储服务器的数据删除方法。
本申请实施例中,存储服务器可以根据该目标时间点确定待删除数据,并删删除所确定的待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令的方式而言,删除一个数据块,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
在本申请的又一实施例中,还提供了一种应用程序,该应用程序用于在运行时执行上述实施例中任一所述的应用于管理服务器的数据删除方法。
本申请实施例中,通过运行上述应用程序而进行数据删除时,管理服务器在确定目标时间点后,将所确定的目标时间点发送给存储服务器,使得存储服务器可以根据该目标时间点删除待删除数据,具体是将在目标时间点之 前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令,删除一个数据块的方式而言,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
在本申请的又一实施例中,还提供了一种应用程序,该应用程序用于在运行时执行上述实施例中任一所述的应用于存储服务器的数据删除方法。
本申请实施例中,通过运行上述应用程序而进行数据删除时,存储服务器可以根据该目标时间点确定待删除数据,并删删除所确定的待删除数据,具体是将在目标时间点之前存储的数据均删除,也就是可以删除多个数据块,相较于相关技术中存储服务器接收到一个删除指令的方式而言,删除一个数据块,可以减少管理服务器与存储服务器之间的网络交互而消耗的时间,减少存储服务器等待指令的时间,提高了数据删除速度,进而提高存储空间释放的效率。
对于应用于分布式存储***的数据删除方法/分布式存储***/数据删除装置/管理服务器/存储服务器/计算机可读存储介质及应用程序实施例而言,由于其基本相似于对应的方法实施例,所以描述的比较简单,相关之处参见图1-图4方法实施例的部分说明即可。
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同 之处。
以上所述仅为本申请的较佳实施例而已,并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等,均包含在本申请的保护范围内。

Claims (33)

  1. 一种数据删除方法,其特征在于,应用于分布式存储***中的管理服务器,所述方法包括:
    当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;
    将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间点删除待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
  2. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
    所述存储时间为存储服务器存储数据时的标准时间;
    所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
  3. 根据权利要求2所述的方法,其特征在于,所述定时根据各个存储服务器的***时间,获得标准时间,包括:
    定时获取各个存储服务器的***时间;
    判断所获得的所有***时间的个数是否大于预设数量;
    如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
    或,
    计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
    如果否,计算所获得的所有***时间的平均值,作为标准时间。
  4. 根据权利要求1所述的方法,其特征在于,所述根据预设的数据删除规则,确定目标时间点,包括:
    根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;
    在所述将所确定的目标时间点发送给存储服务器之前,所述方法还包括:
    获取每个第一分区的分区标识;
    所述将所确定的目标时间点发送给存储服务器,以使接收到所述目标时间点的存储服务器按照所述目标时间删除待删除数据,包括:
    将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器,以使接收到第一分区的目标时间点和对应的分区标识的存储服务器,按照第一分区的目标时间点和对应的分区标识,删除对应第一分区中所存储的待删除数据。
  5. 根据权利要求4所述的方法,其特征在于,所述根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
    获得当前时间;
    针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
  6. 根据权利要求4所述的方法,其特征在于,所述根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
    根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;
    针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;
    针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
  7. 根据权利要求1所述的方法,其特征在于,在根据预设的数据删除规 则,确定目标时间点之后,所述方法还包括:
    删除在所述目标时间点之前存储的数据的索引信息。
  8. 根据权利要求7所述的方法,其特征在于,所述删除在所述目标时间点之前存储的数据的索引信息,包括:
    针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
  9. 一种数据删除方法,其特征在于,应用于分布式存储***中的存储服务器,所述方法包括:
    接收管理服务器发送的目标时间点,其中,所述目标时间点是所述管理服务器确定需要批量删除数据时,根据预设的数据删除规则确定的;
    根据所述目标时间点,确定待删除数据;
    删除所确定的待删除数据。
  10. 根据权利要求9所述的方法,其特征在于,在接收管理服务器发送的目标时间点之前,所述方法还包括:
    接收所述管理器发送的第一分区的分区标识;
    接收管理服务器发送的目标时间点,包括:
    接收管理服务器发送的第一分区的目标时间点;
    所述根据目标时间点确定待删除数据的步骤,包括:
    针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;
    如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;
    所述删除所确定的待删除数据的步骤,包括:
    删除所确定的每一第一分区的待删除数据。
  11. 根据权利要求9所述的方法,其特征在于,所述方法还包括:
    定时向管理服务器发送所述存储服务器自身的***时间,以使所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间。
  12. 一种数据删除方法,其特征在于,应用于分布式存储***,所述分布式存储***包括管理服务器和存储服务器,所述方法包括:
    所述管理服务器当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器;
    所述存储服务器在将接收到目标时间点后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
  13. 根据权利要求12所述的方法,其特征在于,所述存储服务器定时向所述管理服务器发送所述存储服务器自身的***时间;
    所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
    所述存储时间为存储服务器存储数据时的标准时间;
    所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
  14. 根据权利要求13所述的方法,其特征在于,所述管理服务器获得对各个存储服务器的***时间进行同步处理后的时间,作为标准时间,包括:
    定时获取各个存储服务器的***时间;
    判断所获得的所有***时间的个数是否大于预设数量;
    如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
    或,
    计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
    如果否,计算所获得的所有***时间的平均值,作为标准时间。
  15. 根据权利要求12所述的方法,其特征在于,所述管理服务器根据预设的数据删除规则,确定目标时间点,包括:
    根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;
    所述管理服务器在所述将所确定的目标时间点发送给存储服务器之前,获取每个第一分区的分区标识;
    所述管理服务器将所确定的目标时间发送给存储服务器,包括:
    将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器;
    所述存储服务器接收所述管理服务器发送的第一分区的目标时间点和对应的分区标识;
    所述存储服务器根据目标时间点确定待删除数据,包括:
    针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;
    如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;
    所述存储服务器删除所确定的待删除数据,包括:
    删除所确定的每一第一分区的待删除数据。
  16. 根据权利要求15所述的方法,其特征在于,所述管理服务器根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
    获得当前时间;
    针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期之差,作为该第一分区的目标时间点。
  17. 根据权利要求15所述的方法,其特征在于,所述管理服务器根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间 点,包括:
    根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;
    针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;
    针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
  18. 根据权利要求12所述的方法,其特征在于,所述管理服务器删除在所述目标时间点之前存储的数据的索引信息。
  19. 根据权利要求18所述的方法,其特征在于,所述管理服务器删除在所述目标时间点之前存储的数据的索引信息,包括:
    针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
  20. 一种分布式存储***,其特征在于,所述***包括管理服务器和存储服务器,
    所述管理服务器,用于当确定需要批量删除数据时,根据预设的数据删除规则,确定目标时间点;将所确定的目标时间点发送给所述存储服务器;
    所述存储服务器,用于在接收到所述目标时间点后,根据所述目标时间点,确定待删除数据;删除所确定的待删除数据,其中,所述待删除数据为存储时间在所述目标时间点之前的已存储数据。
  21. 根据权利要求20所述的***,其特征在于,所述存储服务器,用于定时向所述管理服务器发送所述存储服务器自身的***时间;
    所述管理服务器,用于定时根据各个存储服务器的***时间,获得标准时间,其中,所述标准时间为对各个存储服务器的***时间进行同步处理后的时间;
    所述存储时间为存储服务器存储数据时的标准时间;
    所述目标时间点为所述待删除数据的存储时间中最晚的存储时间。
  22. 根据权利要求21所述的***,其特征在于,所述管理服务器定时根据各个存储服务器的***时间,获得标准时间,包括:
    定时获取各个存储服务器的***时间;
    判断所获得的所有***时间的个数是否大于预设数量;
    如果是,将除一个最大***时间和一个最小***时间之外的其他所有***时间按照大小进行排序,将排序排在中间的***时间,作为标准时间,
    或,
    计算除一个最大***时间和一个最小***时间之外的其他所有***时间的平均值,作为标准时间;
    如果否,计算所获得的所有***时间的平均值,作为标准时间。
  23. 根据权利要求20所述的***,其特征在于,所述管理服务器用于根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点;获取每个第一分区的分区标识;将每个第一分区的目标时间点及对应的分区标识发送给各个第一分区对应的存储服务器;
    所述存储服务器,用于接收所述管理服务器发送的第一分区的目标时间点和对应的分区标识;针对每一第一分区,根据该第一分区的分区标识判断所述存储服务器是否存储有该第一分区对应的数据;如果是,则将该第一分区在该第一分区的目标时间点之前已存储的数据,确定为该第一分区的待删除数据;删除所确定的每一第一分区的待删除数据。
  24. 根据权利要求23所述的***,其特征在于,所述管理服务器,用于根据预设的每个分区的数据删除规则,确定每个要删除数据的第一分区的目标时间点,包括:
    获得当前时间;
    针对每个第一分区,将所获得的当期时间与该第一分区预设的存储周期 之差,作为该第一分区的目标时间点。
  25. 根据权利要求23所述的***,其特征在于,所述管理服务器,用于根据每一分区当前的存储量以及针对每一分区预设的存储阈值,确定每一分区的待删除存储量;针对待删除存储量不为零的每个第一分区,根据预先记录的该第一分区已存储数据的数据大小,按照预先记录的该第一分区已存储数据的存储时间的先后顺序,确定该第一分区的待删除数据,直至所确定的所有待删除数据的数据总量与该第一分区的待删除存储量之差在预设数据范围内;针对每个第一分区,从针对该第一分区所确定的所有待删除数据的存储时间中,选择最晚的存储时间,作为该第一分区的目标时间点。
  26. 根据权利要求20所述的***,其特征在于,所述管理服务器,用于删除在所述目标时间点之前存储的数据的索引信息。
  27. 根据权利要求26所述的***,其特征在于,所述管理服务器,用于针对每一第一分区,删除针对该第一分区在该第一分区的目标时间点之前存储的数据的索引信息。
  28. 一种管理服务器,其特征在于,包括:第一处理器、第一通信接口、第一存储器和第一通信总线,其中,所述第一处理器、所述第一通信接口、所述第一存储器通过所述第一通信总线完成相互间的通信;
    所述第一存储器,用于存放计算机程序;
    所述第一处理器,用于执行所述第一存储器上所存放的计算机程序,实现权利要求1-8任一所述的方法步骤。
  29. 一种存储服务器,其特征在于,包括:第二处理器、第二通信接口、第二存储器和第二通信总线,其中,所述第二处理器、所述第二通信接口、所述第二存储器通过所述第二通信总线完成相互间的通信;
    所述第二存储器,用于存放计算机程序;
    所述第二处理器,用于执行所述第二存储器上所存放的计算机程序,实现权利要求9-11任一所述的方法步骤。
  30. 一种计算机可读存储介质,其特征在于,所述存储介质内存储有计 算机程序,所述计算机程序被处理器执行时实现权利要求1-8任一所述的方法步骤。
  31. 一种计算机可读存储介质,其特征在于,所述存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现权利要求9-11任一所述的方法步骤。
  32. 一种应用程序,其特征在于,所述应用程序用于在运行时执行权利要求1-8任一项所述的方法步骤。
  33. 一种应用程序,其特征在于,所述应用程序用于在运行时执行权利要求9-11任一项所述的方法步骤。
PCT/CN2018/107277 2017-09-25 2018-09-25 数据删除方法及分布式存储*** WO2019057193A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710876800.3A CN109558065B (zh) 2017-09-25 2017-09-25 数据删除方法及分布式存储***
CN201710876800.3 2017-09-25

Publications (1)

Publication Number Publication Date
WO2019057193A1 true WO2019057193A1 (zh) 2019-03-28

Family

ID=65809549

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/107277 WO2019057193A1 (zh) 2017-09-25 2018-09-25 数据删除方法及分布式存储***

Country Status (2)

Country Link
CN (1) CN109558065B (zh)
WO (1) WO2019057193A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111552667A (zh) * 2020-04-29 2020-08-18 杭州海康威视***技术有限公司 一种数据删除方法、装置及电子设备

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111399754B (zh) * 2019-09-03 2023-11-03 杭州海康威视***技术有限公司 存储空间的释放方法、装置及分布式***
CN110767291A (zh) * 2019-10-15 2020-02-07 武汉联影医疗科技有限公司 医学图像处理方法、装置和存储介质
CN110851402A (zh) * 2019-10-18 2020-02-28 惠州高盛达科技有限公司 基于嵌入式***的文件删除方法及***
CN111859040B (zh) * 2020-07-17 2022-05-13 苏州浪潮智能科技有限公司 一种数据匹配方法、装置及相关设备
CN113126929B (zh) * 2021-04-23 2022-04-22 重庆紫光华山智安科技有限公司 一种特征数据去重的方法、***、介质和终端
CN113537530B (zh) * 2021-09-17 2021-12-31 中建电子信息技术有限公司 基于智慧社区物联网大数据的智能分析及应用方法
CN114328437B (zh) * 2021-12-29 2024-01-12 苏州浪潮智能科技有限公司 一种历史数据快速删除方法、装置、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799514A (zh) * 2011-05-24 2012-11-28 中兴通讯股份有限公司 一种日志记录管理方法和***
CN104679851A (zh) * 2015-02-12 2015-06-03 广东欧珀移动通信有限公司 一种数据删除方法及终端
CN104702700A (zh) * 2015-03-30 2015-06-10 四川神琥科技有限公司 一种邮件提取方法
CN104821907A (zh) * 2015-03-30 2015-08-05 四川神琥科技有限公司 一种电子邮件处理方法
CN106484906A (zh) * 2016-10-21 2017-03-08 焦点科技股份有限公司 一种分布式对象存储***闪回方法及装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4394493B2 (ja) * 2004-03-24 2010-01-06 株式会社日立製作所 ファイル管理方法、ファイル管理装置、及び、ファイル管理プログラム
CN101232514A (zh) * 2008-01-24 2008-07-30 创新科存储技术(深圳)有限公司 网络附加存储节点的元数据同步方法及网络附加存储节点
CN201726424U (zh) * 2009-08-18 2011-01-26 升东网络科技发展(上海)有限公司 分布式存储***
CN103443757B (zh) * 2012-12-31 2017-12-15 华为技术有限公司 数据擦除方法、装置和***
CN103366573A (zh) * 2013-07-10 2013-10-23 中兴智能交通(无锡)有限公司 一种基于云计算的车辆行驶信息追踪方法及***
CN104639859B (zh) * 2013-11-08 2017-10-27 浙江大华技术股份有限公司 一种视频监控设备及其进行数据同步的方法
CN103700133B (zh) * 2013-12-20 2017-07-18 广东威创视讯科技股份有限公司 三维场景分布式渲染同步刷新方法和***
CN103747276A (zh) * 2013-12-24 2014-04-23 乐视网信息技术(北京)股份有限公司 一种cdn数据删除方法及cdn服务器
CN105306858B (zh) * 2014-05-29 2018-10-16 杭州海康威视***技术有限公司 一种视频数据存储方法和装置
CN105989102A (zh) * 2015-02-12 2016-10-05 广东欧珀移动通信有限公司 一种备份数据的删除方法及装置
CN105095489A (zh) * 2015-08-18 2015-11-25 浪潮(北京)电子信息产业有限公司 一种分布式文件删除方法、装置和***
CN106569733A (zh) * 2015-10-12 2017-04-19 北京国双科技有限公司 缓存数据的处理方法和装置
CN105677240B (zh) * 2015-12-30 2019-04-23 上海联影医疗科技有限公司 数据删除方法及***

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102799514A (zh) * 2011-05-24 2012-11-28 中兴通讯股份有限公司 一种日志记录管理方法和***
CN104679851A (zh) * 2015-02-12 2015-06-03 广东欧珀移动通信有限公司 一种数据删除方法及终端
CN104702700A (zh) * 2015-03-30 2015-06-10 四川神琥科技有限公司 一种邮件提取方法
CN104821907A (zh) * 2015-03-30 2015-08-05 四川神琥科技有限公司 一种电子邮件处理方法
CN106484906A (zh) * 2016-10-21 2017-03-08 焦点科技股份有限公司 一种分布式对象存储***闪回方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111552667A (zh) * 2020-04-29 2020-08-18 杭州海康威视***技术有限公司 一种数据删除方法、装置及电子设备
CN111552667B (zh) * 2020-04-29 2023-11-03 杭州海康威视***技术有限公司 一种数据删除方法、装置及电子设备

Also Published As

Publication number Publication date
CN109558065A (zh) 2019-04-02
CN109558065B (zh) 2020-11-27

Similar Documents

Publication Publication Date Title
WO2019057193A1 (zh) 数据删除方法及分布式存储***
US20220335034A1 (en) Multi-master architectures for distributed databases
CN109739929B (zh) 数据同步方法、装置及***
CN107315761B (zh) 一种数据更新方法、数据查询方法及装置
US10331625B2 (en) Managing sequential data store
WO2016127903A1 (zh) 一种数据同步方法、装置和***
CN110309161B (zh) 一种数据同步方法、装置及服务器
EP3508985B1 (en) Scalable synchronization with cache and index management
JP5686034B2 (ja) クラスタシステム、同期制御方法、サーバ装置および同期制御プログラム
CN111552701B (zh) 确定分布式集群中数据一致性的方法及分布式数据***
CN111049928B (zh) 数据同步方法、***、电子设备及计算机可读存储介质
CN110162270B (zh) 基于分布式存储***的数据存储方法、存储节点及介质
CN112434043B (zh) 一种数据同步方法、装置、电子设备及介质
US20190361607A1 (en) Providing combined data from a cache and a storage device
CN107040576A (zh) 信息推送方法及装置、通讯***
WO2019042174A1 (zh) 在主数据库和备数据库之间的同步方法、数据库***和设备
CN116304390B (zh) 时序数据处理方法、装置、存储介质及电子设备
CN105610917B (zh) 实现***中同步数据修复的方法及***
CN115587118A (zh) 任务数据的维表关联处理方法及装置、电子设备
CN112347143A (zh) 多数据流处理方法、装置、终端及存储介质
CN111880909A (zh) 一种基于分布式的发布数据的方法及装置
US11055259B2 (en) Method and system for deleting obsolete files from a file system
CN113312370B (zh) 信息获取方法、装置、电子设备及存储介质
US9893972B1 (en) Managing I/O requests
WO2021212493A1 (zh) 数据同步方法、装置、数据存储***及计算机可读介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18859408

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18859408

Country of ref document: EP

Kind code of ref document: A1