WO2017101016A1

WO2017101016A1 - Method and apparatus for synchronizing service request of storage node

Info

Publication number: WO2017101016A1
Application number: PCT/CN2015/097431
Authority: WO
Inventors: 于传帅; 张程伟
Original assignee: 华为技术有限公司
Priority date: 2015-12-15
Filing date: 2015-12-15
Publication date: 2017-06-22
Also published as: CN107710165A; CN107710165B

Abstract

The embodiments of the present invention relate to a method and apparatus for synchronizing a service request of a storage node. The method comprises: when a first storage node has a fault within t₁ to t₂ while executing a plurality of service requests, determining a set of service requests, which are recorded in a first log, and of which a starting execution moment is within (t₁-2dt) to (t₂+2dt), as a first service request set, and a set of service requests, which are recorded in a second log, and of which a starting execution moment is within (t₁-dt) to (t₂+dt), as a second service request set, wherein dt is the maximum time span for executing the service requests; and determining that a service request belonging to the second service request set but not belonging to the first service request set is a service request lost by the first storage node during the occurrence of the fault so as to facilitate the first storage node executing the lost service request. By means of the method and apparatus for synchronizing a service request of a storage node in the embodiments of the invention, a lost service request can be obtained accurately, and service requests of a normal storage node and a faulty storage node can be synchronized, so that the throughput of a storage system in executing the service requests is improved.

Description

用于存储节点同步业务请求的方法和装置Method and apparatus for storing node synchronization service requests

技术领域Technical field

本发明涉及通信领域，尤其涉及用于存储节点同步业务请求的方法和装置。The present invention relates to the field of communications, and in particular, to a method and apparatus for storing node synchronization service requests.

背景技术Background technique

在较大规模的存储***中存储节点故障是常态，在存储节点故障时需要保证数据不丢失，这就是高可靠性，还要保证***继续提供服务，这就是高可用性，存储厂商一般采用多副本或纠删码(Erasure code，EC)技术，例如将数据复制多份(多副本)，存储多份来提供高可靠性和高可用性。In a large-scale storage system, storage node failure is the normal state. When the storage node fails, it is necessary to ensure that the data is not lost. This is high reliability, and the system continues to provide services. This is high availability. Storage vendors generally use multiple copies. Or Erasure code (EC) technology, such as copying multiple copies (multiple copies) of data, storing multiple copies to provide high reliability and high availability.

假设现在存储***中，在存储节点故障期间，存储***仍在正常下发业务，此时故障存储节点就丢失了一部分业务请求，需要将故障存储节点与正常存储节点的业务请求同步。It is assumed that the storage system is still delivering services normally during the storage node failure. In this case, the faulty storage node loses part of the service request, and the faulty storage node needs to synchronize with the service request of the normal storage node.

在现有的强一致性存储***存储中，一般采用状态同步机(Replication State Machine，简称“RSM”)机制来提供同步技术。RSM机制的基本思想是各个存储节点是完全同步的，例如假设有put(key1)、put(key2)、del(key1)、put(key3)四个业务请求，各个存储节点都必须先执行put(key1)、再执行put(key2)、再执行del(key1)、再执行put(key3)，并且同时以同样顺序记录log，如果存储节点故障期间丢失了一些业务请求，这些业务请求在日志(log)中必然是紧挨着的、连续的，例如后两个业务请求del(key1)、put(key3)，故障存储节点重启后，通过比对log很容易就找到丢失的业务请求，补齐这两个业务请求就可以完成同步。In the existing strong consistent storage system storage, a State Machine (Replication State Machine (RSM) mechanism is generally used to provide synchronization technology. The basic idea of the RSM mechanism is that each storage node is completely synchronized. For example, if there are four service requests, put(key1), put(key2), del(key1), and put(key3), each storage node must execute put ( Key1), then execute put(key2), then execute del(key1), then execute put(key3), and record the log in the same order at the same time. If some service requests are lost during the storage node failure, these service requests are in the log (log ) must be close and continuous, for example, the last two service requests del (key1), put (key3), after the fault storage node restarts, it is easy to find the lost service request by comparing the log, fill this Two business requests can be synchronized.

但是，这样为了高效的识别故障重启存储节点丢失的业务请求，业务请求必须是顺序执行的，而且需要分配和维护业务请求的序列号，这很大程度上限制了存储***执行业务请求的吞吐量。However, in order to efficiently recover the lost service request of the storage node in order to efficiently identify the fault, the service request must be executed sequentially, and the serial number of the service request needs to be allocated and maintained, which greatly limits the throughput of the storage system to perform the service request. .

发明内容Summary of the invention

本发明提供了一种用于存储节点同步业务请求的方法和装置，能够准确的得到丢失的业务请求，同步正常存储节点与故障存储节点的业务请求，提高了存储***执行业务请求的吞吐量。 The present invention provides a method and apparatus for storing a node synchronization service request, which can accurately obtain a lost service request, synchronize a service request of a normal storage node and a faulty storage node, and improve the throughput of the storage system to perform a service request.

第一方面，提供了一种用于存储节点同步业务请求的方法，该方法应用于包括第一存储节点和第二存储节点的存储***中，该第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，该第一日志用于记录该多个业务请求中该第一存储节点已执行的每个业务请求以及该第一存储节点已执行的每个业务请求的开始执行时刻，该第二存储节点用于按照第二顺序执行该多个业务请求并存储第二日志，该第二日志记用于录该多个业务请求中该第二存储节点已执行的每个业务请求以及该第二存储节点已执行的每个业务请求的开始执行时刻，该方法包括：当该第一存储节点在执行该多个业务请求的t₁至t₂内发生故障时，业务请求同步装置确定第一业务请求集合和第二业务请求集合，该第一业务请求集合为该第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，该第二业务请求集合为该第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，该dt大于或等于该第一存储节点开始执行目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值，该目标业务请求为该第一业务请求集合或该第二业务请求集合中的业务请求；该业务请求同步装置确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求；该业务请求同步装置指示该第二存储节点向该第一存储节点发送该丢失的业务请求。In a first aspect, a method for storing a node synchronization service request is provided, the method being applied to a storage system including a first storage node and a second storage node, the first storage node being configured to execute in a first order The service requests and stores a first log, where the first log is used to record each service request that the first storage node has executed in the multiple service requests and the start execution of each service request that the first storage node has performed At a time, the second storage node is configured to execute the multiple service requests in a second order and store a second log, where the second log is used to record each service that the second storage node has executed in the multiple service requests. The request and the start execution time of each service request that the second storage node has executed, the method comprising: when the first storage node fails within t ₁ to t ₂ of executing the plurality of service requests, the service request is synchronized The device determines a first service request set and a second service request set, where the first service request set is a start execution time recorded in the first log at t ₁ -2dt to t ₂ + a set of service requests within 2dt, the second set of service requests being a set of service requests recorded in the second log from t ₁ -dt to t ₂ +dt, the dt being greater than or equal to the first a difference between a time when the storage node starts to execute the target service request and a time when the second storage node starts to execute the target service request, where the target service request is the service in the first service request set or the second service request set a request that the service request synchronization device determines that the service request belonging to the second service request set and does not belong to the first service request set is a service request lost by the first storage node during a failure; the service request synchronization device indicates the first The second storage node sends the lost service request to the first storage node.

因此，本发明实施例的用于存储节点同步业务请求的方法，存储***中的两个存储节点可以不按照业务请求下发的顺序执行业务请求，在其中一个发生故障时，可以通过查找不同时间段内不属于发生故障的存储节点记录的业务请求但属于正常存储节点记录的业务请求为丢失的业务请求，从而使不同存储节点可以并发或乱序地执行业务请求，而不需要专门的分配和维护业务请求的序列号，并且还能够准确的从正常的存储节点得到丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。Therefore, in the method for storing a node synchronization service request in the embodiment of the present invention, two storage nodes in the storage system may not perform the service request in the order in which the service request is delivered, and may find different time when one of the failures occurs. A service request that is not a service request recorded by the failed storage node but belongs to the normal storage node record is a lost service request, so that different storage nodes can perform service requests concurrently or out of order without special allocation and The serial number of the service request is maintained, and the lost service request can be accurately obtained from the normal storage node, the service request synchronization is completed, and the throughput of the storage system executing the service request is improved.

结合第一方面，在第一方面的一种实现方式中，该业务请求同步装置确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求，包括：该业务请求同步装置分别将该第一业务请求集合和该第二业务请求集合中的业务请求按照业务请求的键进行排序；该业务请求同步装置根据排序的结果比对该第一业务请求集合和该第二业务请求集合中的业务请求，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求，这样，可以避免遍历动作，更快地查找丢失的业务请求。With reference to the first aspect, in an implementation manner of the first aspect, the service request synchronization device determines that the service request that belongs to the second service request set and does not belong to the first service request set is that the first storage node is faulty. The service request lost during the period includes: the service request synchronization device respectively sorts the first service request set and the service request in the second service request set according to a key of the service request; the service request synchronization device is compared according to the sorted result ratio Receiving, by the first service request set and the service request in the second service request set, the second service request set And the service request that does not belong to the first service request set is the lost service request, so that the traversal action can be avoided, and the lost service request can be searched more quickly.

结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该业务请求同步装置确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求，包括：该业务请求同步装置根据该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的标志位图；该业务请求同步装置根据该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的标志位图；该业务请求同步装置比对该第一业务请求集合对应的标志位和该第二业务请求集合对应的标志位，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求，这样可以更加直观地确定丢失的业务请求。In conjunction with the first aspect and the foregoing implementation manner, in another implementation manner of the first aspect, the service request synchronization device determines, that the service request that belongs to the second service request set and does not belong to the first service request set is the first a service request lost by a storage node during a failure, the: the service request synchronization device determines a flag bitmap of the first service request set according to a key of each service request in the first service request set; the service request is synchronized The device determines, according to a key of each service request in the second service request set, a flag bitmap of the second service request set; the service request synchronization device compares the flag bit corresponding to the first service request set with the second service The corresponding flag bit of the request set is obtained, and the service request belonging to the second service request set and not belonging to the first service request set is the lost service request, so that the lost service request can be determined more intuitively.

结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该业务请求同步装置确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求，包括：该业务请求同步装置根据布鲁姆过滤器bloom filter算法和该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的位图；该业务请求同步装置根据该bloom filter算法和该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的位图；该业务请求同步装置对比该第一业务请求集合的位图和该第二业务请求集合的位图，确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。In conjunction with the first aspect and the foregoing implementation manner, in another implementation manner of the first aspect, the service request synchronization device determines, that the service request that belongs to the second service request set and does not belong to the first service request set is the first a service request lost by a storage node during a failure, comprising: the service request synchronization device determining the first service request set according to a Bloom filter bloom filter algorithm and a key of each service request in the first service request set Bitmap; the service request synchronization device determines a bitmap of the second service request set according to the bloom filter algorithm and a key of each service request in the second service request set; the service request synchronization device compares the first service The bitmap of the request set and the bitmap of the second service request set determine that the service request belonging to the second service request set and not belonging to the first service request set is the lost service request.

结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该第一顺序与该第二顺序不同。In combination with the first aspect and the foregoing implementation manner, in another implementation manner of the first aspect, the first sequence is different from the second sequence.

结合第一方面及其上述实现方式，在第一方面的另一种实现方式中，该业务请求同步装置指示该第二存储节点向该第一存储节点发送该丢失的业务请求，包括：该业务请求同步装置接收该第二存储节点发送的该丢失的业务请求；该业务请求同步装置向该第一存储节点发送该丢失的业务请求。In conjunction with the first aspect and the foregoing implementation manner, in another implementation manner of the first aspect, the service request synchronization apparatus, the second storage node, instructs the second storage node to send the lost service request to the first storage node, including: the service The request synchronization device receives the lost service request sent by the second storage node; the service request synchronization device sends the lost service request to the first storage node.

第二方面，提供了一种用于存储节点同步业务请求的装置，用于执行上述第一方面或第一方面的任意可能的实现方式中的方法。具体地，该装置包括用于执行上述第一方面或第一方面的任意可能的实现方式中的方法的单元。In a second aspect, an apparatus for storing a node synchronization service request is provided for performing the method of any of the first aspect or the first aspect of the first aspect. In particular, the apparatus comprises means for performing the method of any of the above-described first aspect or any of the possible implementations of the first aspect.

第三方面，提供了一种用于存储节点同步业务请求的装置，包括：存储单元和处理器，该存储单元用于存储指令，该处理器用于执行该存储器存储的指令，并且当该处理器执行该存储器存储的指令时，该执行使得该处理器执行第一方面或第一方面的任意可能的实现方式中的方法。A third aspect provides an apparatus for storing a node synchronization service request, including: storing a unit and a processor for storing instructions for executing instructions stored by the memory, and when the processor executes the instructions stored by the memory, the executing causes the processor to perform the first aspect or the first A method in any possible implementation of the aspect.

第四方面，提供了一种计算机可读介质，用于存储计算机程序，该计算机程序包括用于执行第一方面或第一方面的任意可能的实现方式中的方法的指令。In a fourth aspect, a computer readable medium is provided for storing a computer program comprising instructions for performing the method of the first aspect or any of the possible implementations of the first aspect.

附图说明DRAWINGS

为了更清楚地说明本发明实施例的技术方案，下面将对本发明实施例中所需要使用的附图作简单地介绍。In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments of the present invention will be briefly described below.

图1是根据本发明实施例的存储集群的示意图。1 is a schematic diagram of a storage cluster in accordance with an embodiment of the present invention.

图2是根据本发明实施例的用于存储节点同步业务请求的方法的示意性流程图。2 is a schematic flowchart of a method for storing a node synchronization service request according to an embodiment of the present invention.

图3是根据本发明实施例的确定业务请求集合的方法的示意图。3 is a schematic diagram of a method of determining a set of service requests in accordance with an embodiment of the present invention.

图4是根据本发明实施例的确定业务请求集合的方法的另一示意图。4 is another schematic diagram of a method of determining a set of service requests in accordance with an embodiment of the present invention.

图5是根据本发明实施例的确定业务请求集合的方法的再一示意图。FIG. 5 is still another schematic diagram of a method of determining a service request set according to an embodiment of the present invention.

图6是根据本发明实施例的用于存储节点同步业务请求的装置的示意性框图。6 is a schematic block diagram of an apparatus for storing a node synchronization service request, in accordance with an embodiment of the present invention.

图7是根据本发明另一实施例的用于存储节点同步业务请求的装置的示意性框图。FIG. 7 is a schematic block diagram of an apparatus for storing a node synchronization service request according to another embodiment of the present invention.

具体实施方式detailed description

下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚地描述，显然，所描述的实施例是本发明的一部分实施例，而不是全部实施例。The technical solutions in the embodiments of the present invention will be clearly described in conjunction with the drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, and not all embodiments.

如图1所示，图1为一个存储集群的物理结构，而key-value存储节点可以为一个如图1所示的提供key-value接口的存储集群，该存储集群可以包括多个以太(Ethernet)协议接口硬盘，又称为IP盘，例如图1所示的存储集群包括两个IP盘，或者也可以包括多于两个IP盘。As shown in FIG. 1 , FIG. 1 is a physical structure of a storage cluster, and the key-value storage node may be a storage cluster that provides a key-value interface as shown in FIG. 1 , and the storage cluster may include multiple Ethernets (Ethernet). A protocol interface hard disk, also known as an IP disk. For example, the storage cluster shown in FIG. 1 includes two IP disks, or may include more than two IP disks.

另外，key-value存储节点也可以是单个的提供key-value接口的IP的盘；该key-value存储节点也可以是部署在集群或单盘中的提供key-value接口的虚拟设备。In addition, the key-value storage node may also be a single IP disk that provides a key-value interface; the key-value storage node may also be a key-value interface deployed in a cluster or a single disk. Virtual device.

而本发明实施例描述了一种存储节点同步业务请求的方法，该方法应用于存储***中，该存储***可以包括两个如上述所述的key-value存储节点。在该存储***中，业务请求可以并发的、乱序的执行，每个key-value存储节点会分别保存各自存储节点的key-value数据以及各自存储节点对应的日志，也就是该存储***中会保存两份key-value数据，同时保存两个存储节点对应的日志。当存储***中的存储节点1故障重启后，由于该存储节点中业务请求是可以乱序的、并发的执行，因此存储节点1丢失的业务请求在该存储节点对应的日志(log)1中并不是紧挨着的、连续的，需要高效的查找这些丢失的业务请求，因此，本发明实施例描述了一种存储节点同步业务请求的方法，能够准确的得到故障的存储节点丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。The embodiment of the present invention describes a method for storing a node synchronization service request. The method is applied to a storage system, and the storage system may include two key-value storage nodes as described above. In the storage system, the service request can be executed concurrently and out of order. Each key-value storage node stores the key-value data of the storage node and the log corresponding to the storage node, that is, the storage system Save two key-value data and save the logs corresponding to the two storage nodes. After the storage node 1 in the storage system is restarted, the service request of the storage node 1 is in the log (log) 1 corresponding to the storage node because the service request in the storage node is executable in an out-of-order manner. The present invention describes a method for storing a node synchronization service request, which can accurately obtain a service request that is lost by the failed storage node, and is not immediately and continuously. Completing the synchronization of service requests improves the throughput of the storage system to perform business requests.

图2示出了根据本发明实施例的用于存储节点同步业务请求的方法100的示意性流程图，该方法100应用于包括第一存储节点和第二存储节点的存储***中，该第一存储节点用于按照第一顺序执行多个业务请求，该第一存储节点存储第一日志，该第一日志用于记录在该多个业务请求中该第一存储节点已执行的每个业务请求以及该每个业务请求的开始执行时刻，该第二存储节点用于按照第二顺序执行该多个业务请求，该第二存储节点存储第一日志，该第二日志用于记录在该多个业务请求中该第二存储节点已执行的每个业务请求以及该每个业务请求的开始执行时刻，该方法100可以由业务请求同步装置执行，该装置可以为一个虚拟模块，该装置位于该存储***中的任意一个存储节点中，例如第一存储节点或第二存储节点，或其它存储节点，本发明实施例并不限于此。如图2所示，该方法100包括：2 shows a schematic flowchart of a method 100 for storing a node synchronization service request, which is applied to a storage system including a first storage node and a second storage node, the first, in accordance with an embodiment of the present invention. The storage node is configured to execute a plurality of service requests in a first order, where the first storage node stores a first log, where the first log is used to record each service request that the first storage node has executed in the multiple service requests And the start execution time of the each service request, the second storage node is configured to execute the plurality of service requests in a second order, the second storage node stores a first log, and the second log is used to record in the multiple In the service request, each service request that the second storage node has performed and the start execution time of the each service request, the method 100 may be performed by a service request synchronization device, and the device may be a virtual module, and the device is located in the storage Any one of the storage nodes in the system, such as the first storage node or the second storage node, or other storage node, is not limited to the embodiment of the present invention. herein. As shown in FIG. 2, the method 100 includes:

S110，当该第一存储节点在执行该多个业务请求的t₁至t₂内发生故障时，业务请求同步装置确定第一业务请求集合和第二业务请求集合，该第一业务请求集合为该第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，该第二业务请求集合为该第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，该dt大于或等于该第一存储节点开始执行目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值，该目标业务请求为该第一业务请求集合或该第二业务请求集合中的业务请求； S110, when the first storage node in the plurality of service execution request t ₁ to t ₂ when a failure occurs, a service request synchronizing means determines a first set and a second service request service request set, the first set for the service request a set of service requests recorded in the first log at t ₁ -2dt to t ₂ +2dt, the second service request set being the start execution time recorded in the second log at t ₁ -dt to a set of service requests within t ₂ +dt, the dt being greater than or equal to a difference between a time at which the first storage node begins to execute the target service request and a time at which the second storage node begins to execute the target service request, the target The service request is a service request in the first service request set or the second service request set;

S120，该业务请求同步装置确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求；S120. The service request synchronization device determines that the service request that belongs to the second service request set and does not belong to the first service request set is a service request that is lost during the fault of the first storage node.

S130，该业务请求同步装置指示该第二存储节点向该第一存储节点发送该丢失的业务请求。S130. The service request synchronization device instructs the second storage node to send the lost service request to the first storage node.

具体地，第一存储节点按照第一顺序执行多个业务请求、并由第一日志记录已执行的每个业务请求以及对应的开始执行时刻，第二存储节点按照第二顺序也执行该多个业务请求、并由第二日志记录已执行的每个业务请求以及对应的开始执行时刻，当在t₁至t₂时刻内，第一存储节点发生故障，则在第一日志中确定第一业务请求集合，该第一业务请求集合包括第一日志中记录的在t₁-2dt至t₂+2dt时刻内执行的业务请求；还在第二日志中确定第二业务请求集合，该第二业务请求集合包括第二日志中记录的在t₁-dt至t₂+dt时刻内执行的业务请求，其中，该dt可以称为最大时间跨度，dt大于或等于第一存储节点开始执行目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值，该目标业务请求为***下发的多个业务请求中的任意一个业务请求，可选地，对于该目标业务可以选择属于第一业务请求集合或第二业务请求集合的全部业务请求中的一个，对于该第一业务请求集合和第二业务请求集合包括的全部业务请求均可以满足最大时间跨度dt的范围。再将属于第二业务请求集合但是不属于第一业务请求集合的业务请求确定为第一存储节点在故障期间内丢失的业务请求，即该丢失的业务请求为第一存储节点由于故障而未执行的业务请求，以便于第一存储节点执行该丢失的业务请求。Specifically, the first storage node executes a plurality of service requests in a first order, and records, by the first log, each service request that has been executed and a corresponding start execution time, and the second storage node also executes the plurality of times according to the second sequence. service request, has been performed by the second log record for each service request and corresponding start execution time, when the time t ₂ to t _1, a first storage node fails, it is determined that the first service in the first log a request set, the first service request set includes a service request executed in a first log at a time t ₁ -2dt to t ₂ +2dt; and a second service request set is also determined in the second log, the second service The request set includes a service request executed in a second log at a time t ₁ -dt to t ₂ +dt, wherein the dt may be referred to as a maximum time span, and dt is greater than or equal to the first storage node to start performing a target service request The difference between the time of the second storage node and the time when the second storage node starts to execute the target service request, where the target service request is any one of multiple service requests sent by the system Requesting, optionally, selecting, for the target service, one of all service requests belonging to the first service request set or the second service request set, and all service requests included in the first service request set and the second service request set Can meet the range of the maximum time span dt. And determining, by the service request that belongs to the second service request set but not belonging to the first service request set, the service request that is lost by the first storage node during the fault period, that is, the lost service request is not executed by the first storage node due to the fault. Service request so that the first storage node performs the lost service request.

因此，本发明实施例中的用于存储节点同步业务请求的方法，存储***中的两个存储节点可以不按照业务请求下发的顺序执行业务请求，在其中一个发生故障时，可以通过查找不同时间段内不属于发生故障的存储节点记录的业务请求但属于正常存储节点记录的业务请求为丢失的业务请求，从而使不同存储节点可以并发或乱序地执行业务请求，而不需要专门的分配和维护业务请求的序列号，并且还能够准确的从正常的存储节点得到丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。Therefore, in the method for storing a node synchronization service request in the embodiment of the present invention, two storage nodes in the storage system may not perform the service request in the order in which the service request is delivered, and when one of the failures occurs, the search may be different. The service request that is not recorded by the failed storage node during the time period but belongs to the normal storage node record is a lost service request, so that different storage nodes can execute the service request concurrently or out of order without special allocation. And maintaining the serial number of the service request, and also being able to accurately obtain the lost service request from the normal storage node, complete the service request synchronization, and improve the throughput of the storage system to perform the service request.

在本发明实施例中，对于存储***下发的多个业务请求，第一存储节点可以按照第一顺序执行该多个业务请求，第二存储节点按照第二顺序执行该多个业务请求，该第一顺序和第二顺序可以相同，也可以不同，执行多个业务请求的顺序可以与下发顺序不一致，可以并发或乱序执行多个业务请求，这样可以避免需要专门分配和维护业务请求的序列号，从而导致的限制了业务请求吞吐量。此外，对于每个存储节点记录执行的业务请求，可以通过第一日志记录第一存储节点已执行的业务请求以及该业务请求的开始执行时刻，通过第二日志记录第二存储节点已执行的业务请求以及该业务请求的开始执行时刻。In the embodiment of the present invention, for a plurality of service requests sent by the storage system, the first storage node may perform the multiple service requests in a first order, and the second storage node performs the second sequence in the second order. For multiple service requests, the first sequence and the second sequence may be the same or different. The order of executing multiple service requests may be inconsistent with the delivery order, and multiple service requests may be executed concurrently or in an out-of-order manner. The serial number of the service request is assigned and maintained, which limits the service request throughput. In addition, for each storage node to record the executed service request, the first log records the service request that has been executed by the first storage node and the start execution time of the service request, and records the executed service of the second storage node by using the second log. The request and the start of execution of the business request.

在本发明实施例中，无论第一存储节点还是第二存储节点，都对业务请求的执行顺序不进行限制，但可以对业务请求的执行时间进行限制，如果超过预设时间存储节点未执行业务请求，即在一定时间内，存储***没有收到该存储节点在执行完该业务请求后返回的响应消息，则***可以重新下发该业务请求，该存储节点应该重新执行该业务请求。可选地，这里的预设时间可以根据实际应用进行设置，预先设置一个时间后，若***超过该预设时间未收到该存储节点的响应消息，则***可以重新下发该业务请求，该存储节点可以重新执行该业务请求。In the embodiment of the present invention, the first storage node or the second storage node does not limit the execution order of the service request, but may limit the execution time of the service request. If the storage node does not perform the service when the preset time is exceeded. The request, that is, the storage system does not receive the response message returned by the storage node after executing the service request within a certain period of time, the system may re-issue the service request, and the storage node should re-execute the service request. Optionally, the preset time may be set according to the actual application. After the time is set in advance, if the response message of the storage node is not received, the system may re-issue the service request. The storage node can re-execute the service request.

在本发明实施例中，可以定义最大时间跨度，即当任意一个业务请求，例如目标业务请求在t时刻被执行，出现在一个日志中，也就是由一个存储节点开始执行，那么在t-dt至t+dt时刻这段时间内，该目标业务请求也一定会出现在另一日志中，由该另一个存储节点也开始执行，这里的dt即为最大时间跨度。例如，在t时刻，目标业务请求出现在第一日志中，也就是由第一存储节点开始执行，那么在t-dt至t+dt时刻这段时间内，该业务请求一定会出现在第二日志中，由第二存储节点开始执行，也就是，该dt满足大于或等于该第一存储节点开始执行该目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值。可选地，该最大时间跨度dt可以根据实际情况进行预先设置，各个存储节点执行的各个业务请求都满足该最大时间跨度的要求。可选地，该最大时间跨度dt也可以在发生故障后进行计算，算出一段时间内的业务请求的最大时间跨度，本发明实施例并不限于此。In the embodiment of the present invention, a maximum time span may be defined, that is, when any one of the service requests, for example, the target service request is executed at time t, appears in a log, that is, is executed by a storage node, then at t-dt During the time period t+dt, the target service request must also appear in another log, and the other storage node also starts to execute, where dt is the maximum time span. For example, at time t, the target service request appears in the first log, that is, the first storage node starts execution, then the service request must appear in the second time during the period from t-dt to t+dt. The log is executed by the second storage node, that is, the dt satisfies a time greater than or equal to a time when the first storage node starts executing the target service request and a time when the second storage node starts executing the target service request. Difference. Optionally, the maximum time span dt may be preset according to actual conditions, and each service request performed by each storage node meets the requirement of the maximum time span. Optionally, the maximum time span dt may also be calculated after the fault occurs, and the maximum time span of the service request in a period of time is calculated. The embodiment of the present invention is not limited thereto.

在S110中，当该第一存储节点在执行该多个业务请求的过程中，在t₁至t₂时刻内发生故障时，则根据第一日志确定第一业务请求集合，根据第二日志确定第二业务请求集合，该第一业务请求集合为该第一日志在t₁-2dt至t₂+2dt时刻内记录的已经执行的业务请求的集合，该第二业务请求集合为该第二日志在t₁-dt至t₂+dt时刻内记录的已经执行的业务请求的集合，其中，该dt为最大时间跨度，该dt可以根据第一业务请求集合和第二业务请求集合中包括的业务请求进行计算获得；在S120中，确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求，以便于恢复正常的第一存储节点可以重新执行故障期间丢失的业务请求，从而实现业务请求同步，提高了存储***执行业务请求的吞吐量。In S110, when the first storage node in the plurality of service execution request, _{t. 1} to t in the event of a failure within, the first service time of the first request set ₂ is determined according to the log, the log is determined in accordance with a second a second service request set, where the first service request set is a set of executed service requests recorded in the first log from t ₁ -2dt to t ₂ +2dt, and the second service request set is the second log a set of executed service requests recorded at times t ₁ -dt to t ₂ +dt, wherein the dt is a maximum time span, and the dt may be based on the services included in the first service request set and the second service request set The request is obtained by the calculation; in S120, the service request that belongs to the second service request set and does not belong to the first service request set is a service request that is lost during the failure of the first storage node, so as to restore the normal A storage node can re-execute the lost service request during the failure, thereby implementing service request synchronization and improving the throughput of the storage system to perform service requests.

在本发明实施例中，第一存储节点和第二存储节点在执行业务请求时满足最大时间跨度的要求，当任意存储节点发生故障时，例如，如图3所示，第一存储节点在t₁至t₂时刻内故障，即图3中A段时间内第一存储发生故障，那么这段时间内第一存储节点丢失的业务请求一定属于图3中的B段时间内的业务请求，也就是第二存储节点在t₁-dt至t₂+dt时刻内执行的业务请求，即会出现在第二日志在t₁-dt至t₂+dt时刻内记录的业务请求中，将这些该段时间记录的业务请求确定为第二业务请求集合；并且，丢失的业务请求不属于C段时间，也就是第一存储节点在t₁-2dt至t₂+2dt时刻内执行的业务请求，即不会出现在第一日志中的t₁-2dt至t₂+2dt时刻内记录的业务请求中，将这些该段时间记录的业务请求确定为第一业务请求集合。因此，确定第一存储节点故障期间内丢失的业务请求，也就是确定属于A段时间的业务请求，也就是确定属于第二业务请求集合但是不属于第一业务请求集合的业务请求为第一存储节点丢失的业务请求。In the embodiment of the present invention, the first storage node and the second storage node meet the requirement of the maximum time span when performing the service request. When any storage node fails, for example, as shown in FIG. 3, the first storage node is at t. _The fault occurs in the time from ₁ to t ₂ , that is, the first storage fails during the period A in Figure 3, and the service request lost by the first storage node during this period must belong to the service request in the B period in Figure 3, That is, the service request executed by the second storage node in the time t ₁ -dt to t ₂ +dt, that is, in the service request recorded in the second log from t ₁ -dt to t ₂ +dt, these will be The service request recorded in the segment time is determined as the second service request set; and the lost service request does not belong to the C segment time, that is, the service request executed by the first storage node in the time t ₁ -2dt to t ₂ +2dt, that is, does not appear in the first log t ₁ -2dt to t ₂ + 2dt service request within the time recorded, the service requests the time period is determined to record a first set of service request. Therefore, the service request lost during the failure of the first storage node is determined, that is, the service request belonging to the A-segment time is determined, that is, the service request that belongs to the second service request set but does not belong to the first service request set is the first storage. A business request that the node is missing.

可选地，作为一个实施例，当第一存储节点在t₁至t₂时刻内发生故障时，确定该第一存储节点故障器件丢失的业务请求，也就是确定属于第二业务请求集合，但不属于第一业务请求集合的业务请求，第一业务请求集合为第一存储节点在t₁-2dt至t₂+2dt时刻内执行的业务请求，记录在第一日志中的t₁-2dt至t₂+2dt时刻内；第二业务请求结合为第二存储节点在t₁-dt至t₂+dt时刻内执行的业务请求，记录在第二日志中的t₁-dt至t₂+dt时刻内。具体地，可以通过将第一业务请求集合和第二业务请求结合中的业务请求按照键进行排序，从而确定出丢失的业务请求。例如如图4所示，A、B和C中分别包括多个业务请求，这里的业务请求为以输入键(key)为例，例如，put(key1)，序号1对应该业务请求的键，另外，A对应的为丢失的业务请求，B对应第二业务请求集合，C对应第一业务请求集合，确定属于第二业务请求集合但不属于第一业务请求集合的业务请求，即确定属于B但不属于C的业务请求，可以先将B和C中的业务请求按照键进行排序，排序结果如图4所示，按照键的顺序进行对比可知，属于B但不属于C的业务请求分别为key2、key4、key6和key7，与A中的一致，说明通过这种方式可以准确地判断丢失的业务请求。采用这种排序的方式对比，可以避免遍历日志中记录的每个业务请求，例如对于B中的key6，在查找其是否属于C时，可以发现C中只包括key5，之后是key8，则不但可以确定出key6是丢失数据，也可以确定key7也一定是丢失数据，而不必再次判断key7，避免遍历B中每个业务请求的操作。Alternatively, as an embodiment, when a fault occurs in the first storage node t ₁ to time t _2, the failure determining device of the first storage node is lost service request is determined to belong to a second set of the service request, but The service request that is not in the first service request set, the first service request set is a service request that is performed by the first storage node in the time t ₁ -2dt to t ₂ +2dt, and is recorded in t ₁ -2dt in the first log to t ₂ + 2dt time; the second service request is combined with the service request executed by the second storage node in the time t ₁ -dt to t ₂ +dt, recorded in the second log t ₁ -dt to t ₂ +dt In time. Specifically, the lost service request may be determined by sorting the service request in the first service request set and the second service request according to the key. For example, as shown in FIG. 4, A, B, and C respectively include a plurality of service requests, where the service request is an input key (for example, put(key1), and the serial number 1 corresponds to a service request key. In addition, A corresponds to a lost service request, B corresponds to a second service request set, C corresponds to the first service request set, and determines a service request that belongs to the second service request set but does not belong to the first service request set, that is, belongs to B. However, if the service request does not belong to C, the service requests in B and C can be sorted according to the key. The sort result is shown in Figure 4. According to the order of the keys, the service requests belonging to B but not belonging to C are respectively Key2, key4, key6, and key7 are consistent with those in A, indicating that the lost service request can be accurately determined in this way. By using this sorting method, you can avoid traversing each service request recorded in the log. For example, for key6 in B, if you find that it belongs to C, you can find that only C is included in C, and then key8 is used. It is determined that key6 is missing data, and it can also be determined that key7 must also be lost data, without having to judge key7 again, to avoid traversing the operation of each service request in B.

可选地，作为一个实施例，还可以通过标志位，来确定丢失的业务请求。具体地，例如，如图5所示，将第二业务请求集合中的每个业务请求用标志位表示，即一个业务请求对应一个标志位，确定出第二业务请求集合的标志位图，结果如图5中的B所示，黑色方块表示执行了该标志位对应的业务请求，白色方块表示未执行该标志位对应的业务请求；同样地将第一业务请求集合中的每个业务请求也用标志位表示，结果如图5中的C所示，并且B与C中相同标志位指示的业务请求的键也相同，例如，对于B中的第一标志位，假设该标志位对应的业务请求的键为key1，则在C中的第一标志位对应的业务请求的键也为key1，而B中的第一标志位为白块，表示未执行该key1，C中的第一标志位为黑块，表示已经执行该key1。分别确定第二业务请求集合对应的B与第一业务请求集合对应的C的标志位图后，业务请求同步装置可以根据该标志位图确定属于B但不属于C的丢失的业务请求A，具体地，可以通过公式：A＝B-B∩C＝B^(B&C)来计算，其中，&表示与，^表示异或，即先将B、C按位与，再将结果与B异或，就可以找到丢失的业务请求A对应的标志位，如图5所示，确定的丢失的业务请求A分别为第四标志位和第六标志位，再反向查找就可以得到具体地的丢失的业务请求，即确定第四标志位和第六标志位对应的业务请求。Optionally, as an embodiment, the lost service request may also be determined by using a flag bit. Specifically, for example, as shown in FIG. 5, each service request in the second service request set is represented by a flag bit, that is, one service request corresponds to one flag bit, and a flag bitmap of the second service request set is determined, and the result is obtained. As shown by B in FIG. 5, the black square indicates that the service request corresponding to the flag bit is executed, and the white square indicates that the service request corresponding to the flag bit is not executed; similarly, each service request in the first service request set is also The result is represented by a flag bit, and the result is as shown by C in FIG. 5, and the key of the service request indicated by the same flag bit in B and C is also the same. For example, for the first flag bit in B, the service corresponding to the flag bit is assumed. If the requested key is key1, the key of the service request corresponding to the first flag in C is also key1, and the first flag in B is a white block, indicating that the first flag in C is not executed. A black block indicates that the key1 has been executed. After determining the flag bitmap of the C corresponding to the first service request set by the B corresponding to the second service request set, the service request synchronization apparatus may determine, according to the flag bitmap, the lost service request A that belongs to B but does not belong to C. Ground, can be calculated by the formula: A=BB∩C=B^(B&C), where & indicates that and ^ means XOR, that is, bit and C are first bitwise, and then the result is XORed with B. The flag bit corresponding to the lost service request A can be found. As shown in FIG. 5, the determined lost service request A is the fourth flag bit and the sixth flag bit respectively, and then the reverse search can obtain the specifically lost service. The request determines the service request corresponding to the fourth flag bit and the sixth flag bit.

可选地，作为一个实施例，还可以通过布鲁姆过滤器(bloom filter)算法，来确定丢失的业务请求。具体地，确定了第一业务请求集合和第二业务请求集合中的业务请求，以业务请求为bloom filter为例，则以key为输入，例如第一业务请求集合包括输入key3，则以key3为输入；通过一个函数计算出输入值对应的输出数值，例如通过任意一个哈希(hash)函数h(x)，以 key3为输入，算出输出值h(key3)的值；再取该输出值的任意几位数值，例如取该输出值h(key3)的后两位，根据该两位数在第一业务请求集合的位图中确定出属于该key3的位置，将该位置处的bit设置为1。依此类推，分别将第一业务请求集合和第二业务请求集合中的每个业务请求为输入，算出每个业务请求在各自位图中的位置，并在对应位图中表示出每个业务请求，这样，可以分别确定出第一业务请求集合的位图和第二业务请求集合的位图，将该两个位图进行对比，用A表示需要求得第一存储节点丢失的业务请求，用B表示第二业务请求集合的业务请求，用C表示第一业务请求集合的业务请求，即查找属于B但不属于C的丢失的业务请求A可以通过公式：A＝B-B∩C＝B^(B&C)来计算，其中，&表示与，^表示异或，即先将B、C按位与，再将结果与B异或，这样就可以找到丢失的业务请求A对应的位图的位置。再通过函数反向计算就可以得到该位置对应的业务请求即为丢失的业务请求；或者在生成位图时，同时记录每个为1的bit对应的业务请求，则可以通过该记录的关系，直接确定出丢失的业务请求A。Alternatively, as an embodiment, the lost service request may also be determined by a bloom filter algorithm. Specifically, the service request in the first service request set and the second service request set is determined. If the service request is a bloom filter, the key is input. For example, if the first service request set includes the input key3, the key3 is Input; calculate the output value corresponding to the input value through a function, for example, by any hash function h(x) Key3 is an input, and the value of the output value h(key3) is calculated; and any number of values of the output value is taken, for example, the last two digits of the output value h(key3) are taken, and the first service request set is obtained according to the two digits. The position belonging to the key 3 is determined in the bitmap, and the bit at the position is set to 1. And so on, each service request in the first service request set and the second service request set is input as input, the position of each service request in the respective bitmap is calculated, and each service is represented in the corresponding bitmap. The request is such that the bitmap of the first service request set and the bitmap of the second service request set are respectively determined, and the two bitmaps are compared, and A is used to indicate that the first storage node needs to be lost. B indicates the service request of the second service request set, and C indicates the service request of the first service request set, that is, the lost service request A belonging to B but not belonging to C can pass the formula: A=BB∩C=B^ (B&C) to calculate, where & indicates that and ^ means XOR, that is, bit and C are first bitwise, and then the result is XORed with B, so that the position of the bitmap corresponding to the lost service request A can be found. . Then, through the inverse calculation of the function, the service request corresponding to the location is obtained as a lost service request; or when the bitmap is generated, and the service request corresponding to each bit of 1 is simultaneously recorded, the relationship of the record can be adopted. The lost business request A is directly determined.

在S130中，该业务请求同步装置指示该第二存储节点向该第一存储节点发送该丢失的业务请求。具体地，该业务请求同步装置根据第一业务请求集合和第二业务请求集合，可以确定在第二存储节点的第二日志中，哪些业务请求属于第一存储节点在第一存储节点故障期间未执行的业务请求，这些业务请求即为第一存储节点丢失的业务请求。该业务请求同步装置可以指示第二存储节点将该丢失的业务请求发送至第一存储节点，然后由第一存储节点执行该丢失的业务请求，从而同步业务请求，提高了存储***执行业务请求的吞吐量。In S130, the service request synchronization device instructs the second storage node to send the lost service request to the first storage node. Specifically, the service request synchronization device may determine, according to the first service request set and the second service request set, which service requests belong to the first storage node in the second log of the second storage node during the failure of the first storage node. The service request executed, these service requests are the service requests lost by the first storage node. The service request synchronization device may instruct the second storage node to send the lost service request to the first storage node, and then execute the lost service request by the first storage node, thereby synchronizing the service request, and improving the storage system to perform the service request. Throughput.

在本发明实施例中，该第二存储节点可以根据业务请求同步装置的指示，包括第二存储节点直接将第二日志中记录的属于第一存储节点的丢失的业务请求发送至第一存储节点；或者第二存储节点将该丢失的业务请求发送至该业务请求同步装置，由该业务请求同步装置将丢失的业务请求发送至第一存储节点。或者，第一存储节点在故障恢复后，可以向业务请求同步装置发送请求信息，该请求信息用于请求丢失的业务请求，该业务请求同步装置根据该请求信息指示第二存储节点向第一存储节点发送丢失的业务请求。或者，该业务请求同步装置指示该第二存储节点向该第一存储节点发送该丢失的业务请求包括该业务请求同步装置指示第一存储节点向第二存储节点发送请求信息，该请求信息用于请求丢失的业务请求，第二存储节点根据请求信息向第一存储节点发送丢失的业务请求。或者，第一存储节点向第二存储节点发送请求信息，该请求信息用于请求丢失的业务请求，该业务请求同步装置指示该第二存储节点根据请求信息向该第一存储节点发送该丢失的业务请求包括本发明实施例并不限于此。In the embodiment of the present invention, the second storage node may send, according to the indication of the service request synchronization device, the second storage node to directly send the lost service request belonging to the first storage node recorded in the second log to the first storage node. Or the second storage node sends the lost service request to the service request synchronization device, and the service request synchronization device sends the lost service request to the first storage node. Alternatively, after the fault is recovered, the first storage node may send request information to the service request synchronization device, where the request information is used to request a lost service request, and the service request synchronization device instructs the second storage node to the first storage according to the request information. The node sends a lost service request. Or the service request synchronization device instructs the second storage node to send the lost service request to the first storage node, where the service request synchronization device instructs the first storage node to send to the second storage node Sending request information, the request information is used to request a lost service request, and the second storage node sends the lost service request to the first storage node according to the request information. Or the first storage node sends request information to the second storage node, where the request information is used to request a lost service request, and the service request synchronization device instructs the second storage node to send the lost information to the first storage node according to the request information. The business request including the embodiment of the present invention is not limited thereto.

应理解，在本发明的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be directed to the embodiments of the present invention. The implementation process constitutes any limitation.

因此，本发明实施例中的用于存储节点同步业务请求的方法，存储***中的两个存储节点可以不按照业务请求下发的顺序执行业务请求，在其中一个存储节点发生故障时，可以通过查找不同时间段内不属于发生故障的存储节点记录的业务请求但属于正常存储节点记录的业务请求为丢失的业务请求，从而使不同存储节点可以并发或乱序地执行业务请求，而不需要专门的分配和维护业务请求的序列号，并且还能够准确的从正常的存储节点得到丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。Therefore, in the method for storing a node synchronization service request in the embodiment of the present invention, two storage nodes in the storage system may not perform the service request in the order in which the service request is delivered, and may fail when one of the storage nodes fails. Finding service requests that are not recorded by the failed storage node record in different time periods but the service requests belonging to the normal storage node record are lost service requests, so that different storage nodes can execute service requests concurrently or out of order without specialization The allocation and maintenance of the serial number of the service request, and also can accurately obtain the lost service request from the normal storage node, complete the synchronization of the service request, and improve the throughput of the storage system to perform the service request.

上文中结合图1至图5，详细描述了根据本发明实施例的用于存储节点同步业务请求的方法，下面将结合图6至图7，描述根据本发明实施例的用于存储节点同步业务请求的装置。A method for storing a node synchronization service request according to an embodiment of the present invention is described in detail above with reference to FIG. 1 to FIG. 5, and a storage node synchronization service according to an embodiment of the present invention will be described below with reference to FIG. 6 to FIG. Requested device.

图6示出了根据本发明实施例的用于存储节点同步业务请求的装置200的示意性框图，该装置200可以位于包括第一存储节点和第二存储节点的存储***中，该第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，该第一日志用于记录该多个业务请求中已执行的每个业务请求以及该已执行的每个业务请求的开始执行时刻，该第二存储节点用于按照第二顺序执行该多个业务请求并存储第二日志该第二日志用于记录该多个业务请求中已执行的每个业务请求以及该已执行的每个业务请求的开始执行时刻，该装置200包括：6 shows a schematic block diagram of an apparatus 200 for storing a node synchronization service request, which may be located in a storage system including a first storage node and a second storage node, the first storage, in accordance with an embodiment of the present invention. The node is configured to execute a plurality of service requests in a first order and store a first log, where the first log is used to record each service request that has been executed in the multiple service requests and start execution of each executed business request At a time, the second storage node is configured to execute the plurality of service requests in a second order and store a second log, the second log is used to record each service request that has been executed in the plurality of service requests and each executed At the start of execution of the business request, the apparatus 200 includes:

第一确定单元210，用于当该第一存储节点在执行该多个业务请求的t₁至t₂内发生故障时，确定第一业务请求集合和第二业务请求集合，该第一业务请求集合为该第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，该第二业务请求集合为该第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，该dt大于或等于该第一存储节点开始执行目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值，该目标业务请求为该第一业务请求集合或该第二业务请求集合中的业务请求；The first determining unit 210 is configured to determine, when the first storage node fails within t ₁ to t ₂ of executing the multiple service requests, the first service request set and the second service request set, the first service request The set is a set of service requests recorded in the first log from t ₁ -2dt to t ₂ +2dt, and the second set of service requests is the start execution time recorded in the second log at t ₁ - a set of service requests in dt to t ₂ +dt, the dt being greater than or equal to a difference between a time at which the first storage node begins to execute the target service request and a time at which the second storage node begins to execute the target service request, The target service request is a service request in the first service request set or the second service request set;

第二确定单元220，用于确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求；The second determining unit 220 is configured to determine, by the service request that belongs to the second service request set that does not belong to the first service request set, a service request that is lost by the first storage node during a failure;

处理单元230，用于指示该第二存储节点向该第一存储节点发送该丢失的业务请求。The processing unit 230 is configured to instruct the second storage node to send the lost service request to the first storage node.

因此，本发明实施例中的用于存储节点同步业务请求的装置，可以在存储***中的两个存储节点可以不按照业务请求下发的顺序执行业务请求的情况下，在其中一个发生故障时，通过查找不同时间段内不属于发生故障的存储节点记录的业务请求但属于正常存储节点记录的业务请求为丢失的业务请求，从而使不同存储节点可以并发或乱序地执行业务请求，而不需要专门的分配和维护业务请求的序列号，并且还能够准确的从正常的存储节点得到丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。Therefore, in the embodiment of the present invention, the device for storing the node synchronization service request may perform the service request in the order in which the two storage nodes in the storage system may not be delivered in the order in which the service request is delivered. By searching for service requests that are not recorded by the failed storage node in different time periods but belonging to the normal storage node record, the service request is a lost service request, so that different storage nodes can execute the service request concurrently or out of order without The serial number of the service request is required to be allocated and maintained, and the lost service request can be accurately obtained from the normal storage node, the service request synchronization is completed, and the throughput of the storage system executing the service request is improved.

可选地，该第一确定单元210具体用于：分别将该第一业务请求集合和该第二业务请求集合中的业务请求按照业务请求的键进行排序；根据排序的结果比对该第一业务请求集合和该第二业务请求集合中的业务请求，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, the first determining unit 210 is specifically configured to: separately sort the first service request set and the service request in the second service request set according to a key of the service request; and compare the first result according to the sorted result. The service request set and the service request in the second service request set obtain a service request that belongs to the second service request set and does not belong to the first service request set, and is the lost service request.

可选地，该第一确定单元210具体用于：根据该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的标志位图；该业务请求同步装置根据该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的标志位图；比对该第一业务请求集合对应的标志位图和该第二业务请求集合对应的标志位图，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, the first determining unit 210 is configured to: determine, according to a key of each service request in the first service request set, a flag bitmap of the first service request set; the service request synchronization device is configured according to the second a key of each service request in the service request set, determining a flag bitmap of the second service request set; and obtaining a flag bitmap corresponding to the first service request set and the flag bitmap corresponding to the second service request set, The service request belonging to the second service request set and not belonging to the first service request set is the lost service request.

可选地，该第一确定单元210具体用于：根据布鲁姆过滤器bloom filter算法和该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的位图；该业务请求同步装置根据该bloom filter算法和该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的位图；该业务请求同步装置对比该第一业务请求集合的位图和该第二业务请求集合的位图，确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, the first determining unit 210 is specifically configured to: determine, according to a Bloom filter bloom filter algorithm and a key of each service request in the first service request set, a bitmap of the first service request set; The service request synchronization device determines a bitmap of the second service request set according to the bloom filter algorithm and a key of each service request in the second service request set; the service request is synchronously loaded Comparing the bitmap of the first service request set with the bitmap of the second service request set, determining that the service request belonging to the second service request set and not belonging to the first service request set is the lost service request.

可选地，该第一顺序与该第二顺序可以不同，也可以相同。Optionally, the first sequence may be different from the second sequence or may be the same.

应理解，根据本发明实施例的用于存储节点同步业务请求的装置200可对应于执行本发明实施例中的方法100，并且用于存储节点同步业务请求的装置200中的各个模块的上述和其它操作和/或功能分别为了实现图2中的各个方法的相应流程，为了简洁，在此不再赘述。It should be understood that the apparatus 200 for storing a node synchronization service request according to an embodiment of the present invention may correspond to the above method of performing the method 100 in the embodiment of the present invention, and for storing the node synchronization service request. Other operations and/or functions are respectively implemented in order to implement the corresponding processes of the respective methods in FIG. 2, and are not described herein again for brevity.

如图7所示，本发明实施例还提供了一种用于存储节点同步业务请求的装置300，包括处理器310、存储器320和总线***330，该装置300位于包括第一存储节点和第二存储节点的存储***中，该第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，该第一日志用于记录该多个业务请求中已执行的每个业务请求以及该已执行的每个业务请求的开始执行时刻，该第二存储节点用于按照第二顺序执行该多个业务请求并存储第二日志，该第二日志用于记录该多个业务请求中已执行的每个业务请求以及该已执行的每个业务请求的开始执行时刻。As shown in FIG. 7, an embodiment of the present invention further provides an apparatus 300 for storing a node synchronization service request, including a processor 310, a memory 320, and a bus system 330. The apparatus 300 is located at a first storage node and a second. In the storage system of the storage node, the first storage node is configured to execute a plurality of service requests in a first order and store a first log, where the first log is used to record each service request that has been executed in the multiple service requests and At the start execution time of each executed business request, the second storage node is configured to execute the plurality of service requests in a second order and store a second log, where the second log is used to record that the plurality of service requests have been recorded Each business request executed and the start execution time of each business request that has been executed.

其中，处理器310和存储器320通过总线***330相连，该存储器320用于存储指令，以及存储第一日志和第二日志，该处理器310用于执行该存储器320存储的指令。该存储器320存储程序代码，且处理器310可以调用存储器320中存储的程序代码执行以下操作：当该第一存储节点在执行该多个业务请求的t₁至t₂内发生故障时，确定第一业务请求集合和第二业务请求集合，该第一业务请求集合为该第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，该第二业务请求集合为该第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，该dt大于或等于该第一存储节点开始执行目标业务请求的时刻与该第二存储节点开始执行该目标业务请求的时刻之间的差值，该目标业务请求为该第一业务请求集合或该第二业务请求集合中的业务请求；确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该第一存储节点在发生故障期间丢失的业务请求；指示该第二存储节点向该第一存储节点发送该丢失的业务请求。The processor 310 and the memory 320 are connected by a bus system 330 for storing instructions, and storing a first log and a second log, and the processor 310 is configured to execute the instructions stored by the memory 320. The memory 320 stores program code, and the processor 310 can call the program code stored in the memory 320 to perform the following operations: when the first storage node fails within t ₁ to t ₂ of executing the plurality of service requests, determining a service request set and a second service request set, the first service request set being a set of service requests recorded in the first log starting from t ₁ -2dt to t ₂ +2dt, the second service request And a set of service requests recorded in the second log at a start execution time in t ₁ -dt to t ₂ +dt, where the dt is greater than or equal to a time at which the first storage node starts to execute the target service request and the second a difference between the time when the storage node starts to execute the target service request, the target service request is a service request in the first service request set or the second service request set; determining that the second service request set belongs to the second service request set The service request of the first service request set is a service request lost by the first storage node during a failure; indicating the second storage node to the first storage node Send the missing service request.

应理解，在本发明实施例中，该处理器310可以是中央处理单元(Central Processing Unit，简称为“CPU”)，该处理器310还可以是其他通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。It should be understood that, in the embodiment of the present invention, the processor 310 may be a central processing unit ("CPU"), and the processor 310 may also be other general-purpose processors, digital signal processors (DSPs). , an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and the like. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.

该存储器320可以包括只读存储器和随机存取存储器，并向处理器310提供指令和数据。存储器320的一部分还可以包括非易失性随机存取存储器。例如，存储器320还可以存储设备类型的信息。The memory 320 can include read only memory and random access memory and provides instructions and data to the processor 310. A portion of the memory 320 may also include a non-volatile random access memory. For example, the memory 320 can also store information of the device type.

该总线***330除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线***330。The bus system 330 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 330 in the figure.

在实现过程中，上述方法的各步骤可以通过处理器310中的硬件的集成逻辑电路或者软件形式的指令完成。结合本发明实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器320，处理器310读取存储器320中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 310 or an instruction in a form of software. The steps of the method disclosed in the embodiments of the present invention may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor. The software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like. The storage medium is located in the memory 320, and the processor 310 reads the information in the memory 320 and combines it The hardware completes the steps of the above method. To avoid repetition, it will not be described in detail here.

可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：分别将该第一业务请求集合和该第二业务请求集合中的业务请求按照业务请求的键进行排序；根据排序的结果比对该第一业务请求集合和该第二业务请求集合中的业务请求，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to perform the following operations: respectively, the first service request set and the service request in the second service request set are performed according to the key of the service request. Sorting; according to the sorted result, the service request in the first service request set and the second service request set is obtained, and the service request belonging to the second service request set and not belonging to the first service request set is the lost Business request.

可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：根据该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的标志位图；根据该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的标志位图；比对该第一业务请求集合对应的标志位和该第二业务请求集合对应的标志位，得到属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to perform the following operations: determining, according to a key of each service request in the first service request set, a flag bit of the first service request set. Determining, according to the key of each service request in the second service request set, a flag bitmap of the second service request set; corresponding to the flag bit corresponding to the first service request set and the second service request set The flag bit obtains a service request belonging to the second service request set and does not belong to the first service request set as the lost service request.

可选地，作为一个实施例，处理器310可以调用存储器320中存储的程序代码执行以下操作：根据布鲁姆过滤器bloom filter算法和该第一业务请求集合中每个业务请求的键，确定该第一业务请求集合的位图；根据该bloom filter算法和该第二业务请求集合中每个业务请求的键，确定该第二业务请求集合的位图；对比该第一业务请求集合的位图和该第二业务请求集合的位图，确定属于该第二业务请求集合且不属于该第一业务请求集合的业务请求为该丢失的业务请求。Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to perform the following operations: determining, according to the Bloom filter blow filter algorithm and the key of each service request in the first service request set. a bitmap of the first service request set; determining a bitmap of the second service request set according to the bloom filter algorithm and a key of each service request in the second service request set; comparing bits of the first service request set And a bitmap of the second service request set, determining that the service request belonging to the second service request set and not belonging to the first service request set is the lost service request.

应理解，根据本发明实施例的用于存储节点同步业务请求的装置300可对应于本发明实施例中的用于存储节点同步业务请求的装置200，并可以对应于执行根据本发明实施例的方法100中的相应主体，并且用于存储节点同步业务请求的装置300中的各个模块的上述和其它操作和/或功能分别为了实现图2中的各个方法的相应流程，为了简洁，在此不再赘述。It should be understood that the apparatus 300 for storing a node synchronization service request according to an embodiment of the present invention may correspond to the apparatus 200 for storing a node synchronization service request in the embodiment of the present invention, and may correspond to performing according to an embodiment of the present invention. The above-described and other operations and/or functions of respective modules in the method 100 for storing the node synchronization service request, respectively, in order to implement the respective processes of the various methods in FIG. 2, for the sake of brevity, Let me repeat.

因此，本发明实施例中的用于存储节点同步业务请求的装置，可以在存储***中的两个存储节点可以不按照业务请求下发的顺序执行业务请求的情况下，在其中一个发生故障时，通过查找不同时间段内不属于发生故障的存储节点记录的业务请求但属于正常存储节点记录的业务请求为丢失的业务请求，从而使不同存储节点可以并发或乱序地执行业务请求，而不需要专门的分配和维护业务请求的序列号，并且还能够准确的从正常的存储节点得到丢失的业务请求，完成业务请求同步，提高了存储***执行业务请求的吞吐量。Therefore, in the embodiment of the present invention, the device for storing the node synchronization service request may perform the service request in the order in which the two storage nodes in the storage system may not be delivered in the order in which the service request is delivered. By searching for service requests that are not recorded by the failed storage node in different time periods but belonging to the normal storage node record, the service request is a lost service request, so that different storage nodes can execute the service request concurrently or out of order without Need to specifically allocate and maintain the serial number of the service request, and also be able to accurately get from the normal storage node The loss of the business request, the completion of the business request synchronization, improve the throughput of the storage system to perform business requests.

本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

在本申请所提供的几个实施例中，应该理解到，所揭露的***、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), A variety of media that can store program code, such as random access memory (RAM), disk, or optical disk.

以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。 The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims

一种用于存储节点同步业务请求的方法，其特征在于，所述方法应用于包括第一存储节点和第二存储节点的存储***中，所述第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，所述第一日志用于记录所述多个业务请求中所述第一存储节点已执行的每个业务请求以及所述第一存储节点已执行的每个业务请求的开始执行时刻，所述第二存储节点用于按照第二顺序执行所述多个业务请求并存储第二日志，所述第二日志用于记录所述多个业务请求中所述第二存储节点已执行的每个业务请求以及所述第二存储节点已执行的每个业务请求的开始执行时刻，所述方法包括：A method for storing a node synchronization service request, wherein the method is applied to a storage system including a first storage node and a second storage node, where the first storage node is configured to execute in a first order The service requests and stores a first log, where the first log is used to record each service request that the first storage node has performed in the multiple service requests and each service that the first storage node has performed At the start of execution of the request, the second storage node is configured to execute the plurality of service requests in a second order and store a second log, where the second log is used to record the second of the plurality of service requests Each of the service requests that have been executed by the storage node and the start execution time of each service request that has been executed by the second storage node, the method includes:

当所述第一存储节点在执行所述多个业务请求的t₁至t₂内发生故障时，业务请求同步装置确定第一业务请求集合和第二业务请求集合，所述第一业务请求集合为所述第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，所述第二业务请求集合为所述第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，所述dt大于或等于所述第一存储节点开始执行目标业务请求的时刻与所述第二存储节点开始执行所述目标业务请求的时刻之间的差值，所述目标业务请求为所述第一业务请求集合或所述第二业务请求集合中的业务请求；When the first storage node fails within t ₁ to t ₂ of executing the plurality of service requests, the service request synchronization device determines a first service request set and a second service request set, the first service request set a set of service requests for the start execution time in the first log from t ₁ -2dt to t ₂ +2dt, the second service request set being the start execution time recorded in the second log at t ₁ -dt to t ₂ + dt request traffic within a set, the dt is greater than or equal to the first storage node begins execution time of the target service request time with the second target storage node starts executing the service request a difference between the target service request is a service request in the first service request set or the second service request set;

所述业务请求同步装置确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求；The service request synchronization device determines that the service request that belongs to the second service request set and does not belong to the first service request set is a service request that is lost by the first storage node during a failure;

所述业务请求同步装置指示所述第二存储节点向所述第一存储节点发送所述丢失的业务请求。The service request synchronization device instructs the second storage node to send the lost service request to the first storage node.
根据权利要求1所述的方法，其特征在于，所述业务请求同步装置确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求，包括：The method according to claim 1, wherein the service request synchronization means determines that a service request belonging to the second service request set and not belonging to the first service request set is that the first storage node is occurring Business requests lost during a failure, including:

所述业务请求同步装置分别将所述第一业务请求集合和所述第二业务请求集合中的业务请求按照业务请求的键进行排序；The service request synchronization device sequentially sorts the service request in the first service request set and the second service request set according to a key of the service request;

所述业务请求同步装置根据排序的结果比对所述第一业务请求集合和所述第二业务请求集合中的业务请求，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。And the service request synchronization device compares the service request in the first service request set and the second service request set according to the sorted result, and obtains the second service request set that does not belong to the first service request. The aggregated business request is for the lost business request.
根据权利要求1所述的方法，其特征在于，所述业务请求同步装置确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求，包括：The method of claim 1 wherein said service request synchronization device Determining, by the service request that belongs to the second service request set that does not belong to the first service request set, a service request that is lost during the failure of the first storage node, including:

所述业务请求同步装置根据所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的标志位图；Determining, by the service request synchronization device, a flag bitmap of the first service request set according to a key of each service request in the first service request set;

所述业务请求同步装置根据所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的标志位图；Determining, by the service request synchronization device, a flag bitmap of the second service request set according to a key of each service request in the second service request set;

所述业务请求同步装置比对所述第一业务请求集合的标志位图和所述第二业务请求集合的标志位图，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。The service request synchronization device compares the flag bitmap of the first service request set with the flag bitmap of the second service request set to obtain the second service request set and does not belong to the first service request. The aggregated business request is for the lost business request.
根据权利要求1所述的方法，其特征在于，所述业务请求同步装置确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求，包括：The method according to claim 1, wherein the service request synchronization means determines that a service request belonging to the second service request set and not belonging to the first service request set is that the first storage node is occurring Business requests lost during a failure, including:

所述业务请求同步装置根据布鲁姆过滤器bloom filter算法和所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的位图；The service request synchronization device determines a bitmap of the first service request set according to a Bloom filter and a key of each service request in the first service request set;

所述业务请求同步装置根据所述bloom filter算法和所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的位图；Determining, by the service request synchronization device, a bitmap of the second service request set according to the bloom filter algorithm and a key of each service request in the second service request set;

所述业务请求同步装置对比所述第一业务请求集合的位图和所述第二业务请求集合的位图，确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。The service request synchronization device compares the bitmap of the first service request set with the bitmap of the second service request set, and determines a service that belongs to the second service request set and does not belong to the first service request set. Request for the lost business request.
根据权利要求1至4中任一项所述的方法，其特征在于，所述第一顺序与所述第二顺序不同。The method according to any one of claims 1 to 4, wherein the first order is different from the second order.
一种用于存储节点同步业务请求的装置，其特征在于，所述装置位于包括第一存储节点和第二存储节点的存储***中，所述第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，所述第一日用于志记录所述多个业务请求中所述第一存储节点已执行的每个业务请求以及所述第一存储节点已执行的每个业务请求的开始执行时刻，所述第二存储节点用于按照第二顺序执行所述多个业务请求并存储第二体质，所述第二日志用于记录所述多个业务请求中所述第二存储节点已执行的每个业务请求以及所述第二存储节点已执行的每个业务请求的开始执行时刻，所述装置包括：An apparatus for storing a node synchronization service request, wherein the apparatus is located in a storage system including a first storage node and a second storage node, where the first storage node is configured to execute multiples in a first order The service requests and stores a first log, where the first day is used to record each service request that the first storage node has performed in the multiple service requests and each service that the first storage node has performed At the start of execution of the request, the second storage node is configured to execute the plurality of service requests in a second order and store a second physique, and the second log is configured to record the second of the plurality of service requests The storage execution time of each service request that has been executed by the node and the start of execution of each service request that the second storage node has performed, the device includes:

第一确定单元，用于当所述第一存储节点在执行所述多个业务请求的t₁至t₂内发生故障时，确定第一业务请求集合和第二业务请求集合，所述第一业务请求集合为所述第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，所述第二业务请求集合为所述第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，所述dt大于或等于所述第一存储节点开始执行目标业务请求的时刻与所述第二存储节点开始执行所述目标业务请求的时刻之间的差值，所述目标业务请求为所述第一业务请求集合或所述第二业务请求集合中的业务请求；A first determining unit, when the first storage node when a failure occurs within ₁ to t ₂ performs the service request plurality of t, and determining a second set of a first service request service request set, said first The service request set is a set of service requests recorded in the first log at a start execution time from t ₁ -2dt to t ₂ +2dt, and the second service request set is a start execution of the record in the second log a set of service requests at time t ₁ -dt to t ₂ +dt, the dt being greater than or equal to a time when the first storage node starts to execute a target service request, and the second storage node starts executing the target service a difference between the requested moments, where the target service request is a service request in the first service request set or the second service request set;

第二确定单元，用于确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求；a second determining unit, configured to determine, by the service request that belongs to the second service request set that does not belong to the first service request set, a service request that is lost by the first storage node during a failure;

处理单元，用于指示所述第二存储节点向所述第一存储节点发送所述丢失的业务请求。And a processing unit, configured to instruct the second storage node to send the lost service request to the first storage node.
根据权利要求6所述装置，其特征在于，所述第一确定单元具体用于：The device according to claim 6, wherein the first determining unit is specifically configured to:

分别将所述第一业务请求集合和所述第二业务请求集合中的业务请求按照业务请求的键进行排序；And respectively sorting the service request in the first service request set and the second service request set according to a key of a service request;

根据排序的结果比对所述第一业务请求集合和所述第二业务请求集合中的业务请求，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Obtaining, according to the sorted result, the service request in the first service request set and the second service request set, obtaining a service request that belongs to the second service request set and does not belong to the first service request set. Describe the lost business request.
根据权利要求6所述的装置，其特征在于，所述第一确定单元具体用于：The device according to claim 6, wherein the first determining unit is specifically configured to:

根据所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的标志位图；Determining, according to a key of each service request in the first service request set, a flag bitmap of the first service request set;

根据所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的标志位图；Determining, according to a key of each service request in the second service request set, a flag bitmap of the second service request set;

比对所述第一业务请求集合对应的标志位和所述第二业务请求集合对应的标志位，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Comparing the flag bit corresponding to the first service request set with the flag bit corresponding to the second service request set, obtaining a service request that belongs to the second service request set and does not belong to the first service request set Describe the lost business request.
根据权利要求6所述的装置，其特征在于，所述第一确定单元具体用于：The device according to claim 6, wherein the first determining unit is specifically configured to:

根据布鲁姆过滤器bloom filter算法和所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的位图； Determining a bitmap of the first service request set according to a Bloom filter bloom filter algorithm and a key of each service request in the first service request set;

根据所述bloom filter算法和所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的位图；Determining, according to the bloom filter algorithm and a key of each service request in the second service request set, a bitmap of the second service request set;

对比所述第一业务请求集合的位图和所述第二业务请求集合的位图，确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Comparing the bitmap of the first service request set with the bitmap of the second service request set, determining that the service request belonging to the second service request set and not belonging to the first service request set is the lost Business request.
根据权利要求6至9中任一项所述的装置，其特征在于，所述第一顺序与所述第二顺序不同。The apparatus according to any one of claims 6 to 9, wherein the first order is different from the second order.
一种用于存储节点同步业务请求的装置，其特征在于，所述装置位于包括第一存储节点和第二存储节点的存储***中，所述第一存储节点用于按照第一顺序执行多个业务请求并存储第一日志，所述第一日志用于记录所述多个业务请求中所述第一存储节点已执行的每个业务请求以及所述第一存储节点已执行的每个业务请求的开始执行时刻，所述第二存储节点用于按照第二顺序执行所述多个业务请求并存储第二日志，所述第二日志用于记录所述多个业务请求中所述第二存储节点已执行的每个业务请求以及所述第二存储节点已执行的每个业务请求的开始执行时刻，所述装置包括：存储器和处理器，An apparatus for storing a node synchronization service request, wherein the apparatus is located in a storage system including a first storage node and a second storage node, where the first storage node is configured to execute multiples in a first order The service requests and stores a first log, where the first log is used to record each service request that the first storage node has performed in the multiple service requests and each service request that has been executed by the first storage node The second storage node is configured to execute the plurality of service requests in a second order and store a second log, where the second log is used to record the second storage of the plurality of service requests Each of the service requests that the node has executed and the start execution time of each service request that the second storage node has performed, the device includes: a memory and a processor,

所述存储器用于存储所述第一日志和所述第二日志，The memory is configured to store the first log and the second log,

所述处理器用于：The processor is used to:

当所述第一存储节点在执行所述多个业务请求的t₁至t₂内发生故障时，确定第一业务请求集合和第二业务请求集合，所述第一业务请求集合为所述第一日志中记录的开始执行时刻在t₁-2dt至t₂+2dt内的业务请求的集合，所述第二业务请求集合为所述第二日志中记录的开始执行时刻在t₁-dt至t₂+dt内的业务请求的集合，所述dt大于或等于所述第一存储节点开始执行目标业务请求的时刻与所述第二存储节点开始执行所述目标业务请求的时刻之间的差值，所述目标业务请求为所述第一业务请求集合或所述第二业务请求集合中的业务请求；Determining, when the first storage node fails within t ₁ to t ₂ of executing the plurality of service requests, the first service request set and the second service request set, where the first service request set is the a set of service requests recorded in a log from t ₁ -2dt to t ₂ +2dt, the second service request set being the start execution time recorded in the second log at t ₁ -dt to a set of service requests within t ₂ +dt, the dt being greater than or equal to a difference between a time at which the first storage node begins to execute a target service request and a time at which the second storage node begins to execute the target service request a value, the target service request is a service request in the first service request set or the second service request set;

确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述第一存储节点在发生故障期间丢失的业务请求；Determining, by the service request that belongs to the second service request set and not belonging to the first service request set, a service request that is lost during the failure of the first storage node;

指示所述第二存储节点向所述第一存储节点发送所述丢失的业务请求。Instructing the second storage node to send the lost service request to the first storage node.
根据权利要求11所述装置，其特征在于，所述处理器具体用于：The device according to claim 11, wherein the processor is specifically configured to:

分别将所述第一业务请求集合和所述第二业务请求集合中的业务请求按照业务请求的键进行排序；Separating the first service request set and the service request in the second service request set respectively Sort by the key of the business request;

根据排序的结果比对所述第一业务请求集合和所述第二业务请求集合中的业务请求，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Obtaining, according to the sorted result, the service request in the first service request set and the second service request set, obtaining a service request that belongs to the second service request set and does not belong to the first service request set. Describe the lost business request.
根据权利要求11所述的装置，其特征在于，所述处理器具体用于：The device according to claim 11, wherein the processor is specifically configured to:

根据所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的标志位图；Determining, according to a key of each service request in the first service request set, a flag bitmap of the first service request set;

根据所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的标志位图；Determining, according to a key of each service request in the second service request set, a flag bitmap of the second service request set;

比对所述第一业务请求集合对应的标志位图和所述第二业务请求集合对应的标志位图，得到属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Comparing the flag bitmap corresponding to the first service request set with the flag bitmap corresponding to the second service request set, obtaining a service request that belongs to the second service request set and does not belong to the first service request set For the lost business request.
根据权利要求11所述的装置，其特征在于，所述处理器具体用于：The device according to claim 11, wherein the processor is specifically configured to:

根据布鲁姆过滤器bloom filter算法和所述第一业务请求集合中每个业务请求的键，确定所述第一业务请求集合的位图；Determining a bitmap of the first service request set according to a Bloom filter bloom filter algorithm and a key of each service request in the first service request set;

根据所述bloom filter算法和所述第二业务请求集合中每个业务请求的键，确定所述第二业务请求集合的位图；Determining, according to the bloom filter algorithm and a key of each service request in the second service request set, a bitmap of the second service request set;

对比所述第一业务请求集合的位图和所述第二业务请求集合的位图，确定属于所述第二业务请求集合且不属于所述第一业务请求集合的业务请求为所述丢失的业务请求。Comparing the bitmap of the first service request set with the bitmap of the second service request set, determining that the service request belonging to the second service request set and not belonging to the first service request set is the lost Business request.
根据权利要求11至14中任一项所述的装置，其特征在于，所述第一顺序与所述第二顺序不同。 Apparatus according to any one of claims 11 to 14, wherein said first order is different from said second order.